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ABSTRACT 


Human  Systems  Integration  (HSI)  is  a  process  designed  to  reduce  life- 
cycle  costs  and  improve  system  performance  by  considering  human-related 
domains.  Acquisition  specialists  lack  objective,  quantitative  research  findings  on 
which  to  base  trade-off  analyses.  This  thesis  used  eight  fiscal  years  of  historical 
safety  (mishaps),  manpower  (manning  levels)  and  system  performance  (SORTS) 
data  on  the  U.S.  Navy’s  Frigates,  collected  from  computer  databases  of  routine 
reports,  to  explore  relationships  in  an  existing  notional  model  of  HSI.  Three 
hypotheses  were  tested:  there  is  a  negative  relationship  between  manning  and 
SORTS  levels;  there  is  a  negative  relationship  between  manning  and  mishaps; 
and  there  is  a  positive  relationship  between  mishaps  and  SORTS  levels.  No 
significant  relationships  were  found  between  SORTS  levels  and  manning  or 
mishaps.  When  all  of  the  ships  were  ranked  for  each  month  based  on  percent  of 
total  manning  and  number  of  reportable  mishaps,  a  positive  correlation 
(Spearman’s  rho  =  0.4194,  p-value  =  0.0294)  was  found  corresponding  to  a 
negative  relationship  between  manning  levels  and  mishap  rates.  More  detailed 
research  is  needed  to  isolate  the  relationship  between  manning  levels  and 
mishap  rates  from  numerous  other  influences  and  any  noise  that  may  be  present 
in  the  data  set. 
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EXECUTIVE  SUMMARY 


Human  Systems  Integration  (HSI)  is  a  process  designed  to  reduce  life- 
cycle  costs  and  improve  system  performance  by  considering  human-related 
domains.  All  Department  of  Defense  (DoD)  acquisition  programs  are  required  to 
consider  HSI.  The  Navy’s  HSI  guidance  includes  modernization  programs  of 
legacy  systems.  However,  acquisition  specialists  lack  objective,  quantitative 
research  findings  on  which  to  base  trade-off  analyses  for  developmental  or 
legacy  systems. 

This  thesis  used  eight  fiscal  years  of  historical  safety  (mishaps), 
manpower  (manning  levels)  and  system  performance  (SORTS)  data  on  the  U.S. 
Navy’s  Oliver  Hazard  Perry  (FFG-7)  Class  Frigates,  collected  from  computer 
databases  of  routine  reports,  to  explore  relationships  in  an  existing  notional 
model  of  HSI.  The  three  hypotheses  tested  were,  there  is  a  negative  relationship 
between  manning  and  SORTS  levels,  there  is  a  negative  relationship  between 
manning  and  mishaps,  and  there  is  a  positive  relationship  between  mishaps  and 
SORTS  levels. 

No  significant  relationships  were  found  between  SORTS  levels  and 
manning  or  mishaps.  When  all  of  the  ships  were  ranked  from  1  (best),  to  27 
(worst),  for  each  month  of  the  study  period  based  on  percent  of  total  manning 
and  number  of  reportable  mishaps,  a  positive  correlation  (Spearman’s  rho  = 
0.4194,  p-value  =  0.0294)  was  found  corresponding  to  a  negative  relationship 
between  manning  levels  and  mishap  rates. 

No  practically  significant  differences  were  found  in  manning,  personnel, 
performance  or  safety  based  on  date,  ships,  homeport,  or  region.  This  means 
that  data  collected  on  a  sample  of  ships  should  be  applicable  to  all  Navy 
Frigates.  However,  there  were  significant  differences  found  in  manning  levels 
between  the  Navy  Reserve  ships  and  active  component  ships. 
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The  data  analyzed  for  this  thesis  were  insufficient  to  validate  all  the 
relationships  in  NPS’s  notional  HSI  trade  space  tool  and  more  detailed  research 
is  needed  to  isolate  the  relationship  between  manning  levels  and  mishap  rates 
from  numerous  other  influences  and  any  noise  that  may  be  present  in  the  data 
set. 

HSI,  though  required  by  law,  is  often  confused  as  just  HFE  or  just  SE. 
There  remains  a  need  for  shared  representations  of  the  HSI  trade  space  to 
facilitate  communication  between  what  are  often  isolated  domains.  While  the 
data  collected  for  this  thesis  were  insufficient  to  validate  all  the  relationships  in 
one  notional  trade  space  tool,  finding  a  negative  relationship  between  manning 
and  mishap  rates  is  a  substantial  step  forward.  Just  as  this  thesis  advanced 
previous  work,  further  research  is  still  needed  to  fully  develop  NPS’s  HSI  TST  to 
maturity. 
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I.  INTRODUCTION 


The  stated  goal  of  Human  Systems  Integration  (HSI)  is  to  reduce  life-cycle 
costs  and  improve  system  performance  by  considering  human-related  domains. 
All  Department  of  Defense  (DoD)  acquisition  programs  are  required  to  consider 
HSI.  However,  there  is  a  dearth  of  evidence  to  define  relationships  among 
domains.  This  thesis  is  the  first  empirical  exploration  of  the  relationships  between 
HSI  domains  and  system  performance,  based  on  historical  data. 

A.  HISTORY  OF  HUMAN  SYSTEMS  INTEGRATION  (HSI) 

The  DoD  defines  HSI  as  a  process  designed  to  “optimize  total  system 
performance,  minimize  total  ownership  costs,  and  ensure  that  the  system  is  built 
to  accommodate  the  characteristics  of  the  user  population  that  will  operate, 
maintain,  and  support  the  system”  (Department  of  Defense  Instruction  [DoDI] 
5000.2,  2004,  p.  E7-1).  HSI  encompasses  seven  distinct  domains:  human  factors 
engineering  (HFE),  personnel,  habitability,  manpower,  training,  environmental 
safety  and  occupational  health  (ESOH),  and  survivability  (DoDI  5000.2,  2004). 

At  the  most  basic  level,  human  considerations  are  as  old  as  tool  making; 
older  than  civilization  itself.  Human-centered  design  considerations  have  been 
around  since  the  first  humanoid  chose  which  rock  to  shape  into  a  hand  ax.  Over 
the  centuries,  systems  became  more  complex  and  mass  production  replaced 
custom  production,  increasing  the  challenge  of  designing  systems  to 
accommodate  human  users. 

All  of  the  individual  domains  of  HSI  predate  the  formation  of  HSI  as  a 
discipline.  Academic  books  on  the  subject  describe  HFE  as  the  initial  domain  of 
HSI.  The  addition  of  the  other  domains,  and  the  consideration  of  the  interactions 
between  the  domains,  produces  the  HSI  process.  Research  into  pilot  error  in 
World  War  II  was  instrumental  in  formalizing  work  in  HFE  into  a  discipline  (Pew  & 
Mavor,  2007;  Simpson,  2003).  In  the  foreword  to  the  Handbook  of  Human 
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Systems  Integration,  Booher  describes  HSI  as  the  third  milestone  of 
human-focused  considerations  within  the  U.S.  Defense  community.  He  lists  the 
first  milestone  as  the  development  of  HFE  and  the  second  milestone  as 
implementation  of  the  U.S.  Army’s  Manpower  and  Personnel  Integration 
(MANPRINT)  program  (Booher,  2003). 

B.  RATIONALE  FOR  THE  STUDY 

1.  The  Human  Systems  Integration  (HSI)  Mandate 

In  2003,  the  United  States  Congress,  through  Department  of  Defense 
Directive  (DoDD)  5000.1,  mandated  the  inclusion  of  HSI  in  all  DoD  acquisition 
programs.  The  primary  sources  of  HSI  requirements  are  The  Defense  Acquisition 
System,  DoDD  5000.1,  and  Operations  of  the  Defense  Acquisition  System, 
DoDI  5000.2.  Chapter  Six  of  the  Defense  Acquisition  Guidebook  (DAG)  provides 
nonbinding  guidance  on  implementing  the  mandates  in  DoDD  5000.1  and 
DoDI  5000.2. 

The  most  cost  effective  way  to  implement  the  congressional  mandate  is  to 
consider  HSI  trade-offs  from  the  beginning  of  the  acquisition  process.  However, 
acquisition  specialists  lack  objective,  quantitative  research  findings  on  which  to 
base  trade-off  analyses  for  developmental  or  legacy  systems. 

2.  Human  Systems  Integration  (HSI)  in  Legacy  Systems 

DoDD  5000.1  applies  to  all  New  DoD  acquisition  programs  (DoDD  5000.1, 
2007).  Secretary  of  the  Navy  (SECNAV)  Instruction  5000. 2C  and  Naval  Sea 
Systems  Command  (NAVSEA)  Instruction  3900. 8A  expand  the  applicability  of 
HSI  to  include  modernization,  disposal,  and  demilitarization  of  new  and  existing 
programs.  Both  of  these  Navy  instructions  recognize  the  need  for  HSI  to  evaluate 
alterations  to  legacy  systems.  Navy  ships  are  not  static  objects;  they  are  large 
and  complex  systems  of  systems,  operating  in  a  dynamic  global  environment. 
Predictions  are  that  in  the  next  ten  years,  “HSI  personnel  will  become  the  experts 
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in  issue  tracking  and  resolution  as  systems  are  now  never  finished”  (Pew  & 
Mavor,  2007,  p.  286).  Currently,  program  managers  struggle  to  predict  and  plan 
for  HSI-related  considerations  (total  life-cycle  costs,  utilization,  operational 
environment)  in  the  initial  acquisition  process.  This  thesis  seeks  to  further 
develop  a  model  that  can  evaluate  the  impact  of  alterations  to  developmental  and 
legacy  programs.  Alterations  to  legacy  programs  could  include  any  or  all  of  the 
doctrine,  organization,  training,  materiel,  leadership  and  education,  personnel, 
and  facilities  (DOTMLPF)  factors. 

The  absence  of  attention  to  HSI  throughout  the  full  life  cycle  is  likely  due 
to  the  argument  of  a  low  return  on  investment.  Changes  and  improvements  in 
legacy  systems  are  severely  constrained  by  the  existing  HFE  level.  For  example, 
HFE,  through  physical  space  and  acceptable  habitability  levels,  sets  the  upper 
limit  of  manning  on  a  ship.  Lower  limits  are  set  through  the  tasks  (manpower) 
and  skills  (personnel)  needed  to  operate  the  ship.  This  leaves  HFE  and  training 
as  the  primary  avenues  to  alter  a  system’s  manpower,  personnel,  habitability, 
safety,  and  survivability.  Compared  to  the  initial  concept  development  of  a  novel 
system,  legacy  systems  have  a  restricted  trade  space  and  higher  cost  for 
marginal  improvements. 

The  low-return-on-investment  argument  overlooks  two  opportunities. 
Since  DoD  systems  are  large,  complex,  and  operate  in  a  dynamic  environment, 
they  change  constantly.  Using  HSI  will  improve  the  quality  of  modernization 
efforts.  Second,  identifying  and  capturing  lessons  learned,  even  those  not  cost 
effective  to  fix  in  the  current  system,  will  improve  future  systems  under 
development.  In  addition  to  evaluating  initial  design  decisions  of  legacy  systems, 
program  managers  would  benefit  from  careful  studies  of  how  systems  evolve 
and  operate. 
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c. 


OBJECTIVE 


1.  Scope 

This  thesis  seeks  to  explore  relationships  among  HSI  domains.  A  subset 
of  the  relationships  from  the  Naval  Postgraduate  School’s  (NPS)  Trade-Space 
Tool  (TST)  is  studied,  using  a  retrospective  analysis  of  historical  data.  This  thesis 
is  built  on  a  theoretical  foundation  laid  by  an  NPS  thesis,  “Toward  an  Improved 
Method  of  HSI  Evaluation  in  Defense  Acquisition”  (Simpson,  2006).  The  fleet  of 
U.S.  Navy  Frigates  served  as  a  proof  of  concept  to  explore  the  relationships 
among  HSI  domains. 

2.  Limitations 

This  thesis  focused  on  a  single  class  of  ship  in  an  effort  to  improve 
accuracy  at  the  expense  of  generalization  to  other  platforms.  The  U.S.  Navy’s 
Oliver  Hazard  Perry  Class  Frigate  (FFG-7)  is  an  ideal  initial  platform  to  analyze 
for  several  reasons.  It  is  the  smallest  and  simplest  ship  in  the  cruiser  and 
destroyer  (CRUDES)  class.  The  decision  to  focus  on  the  U.S.  Navy  Frigate 
simplified  the  amount  of  data  to  gather,  encode,  and  analyze.  Two  of  the  four 
surface  ship  classes  currently  in  development  are  in  the  CRUDES  class — the 
Littoral  Combat  Ship  and  the  next  generation  Destroyer.  Minesweepers  and 
patrol  craft,  though  smaller  and  simpler  systems,  might  not  provide  as  much 
insight  into  these  current  systems  under  development. 

The  data  collected  were  limited  to  measures  available  on  computer 
databases  from  routine  reports.  This  study  did  not  attempt  to  collect  data  from 
the  ships  directly,  but  instead  relied  on  data  already  stored  in  a  variety  of 
repositories.  The  complexity  of  the  problem  space  necessitated  numerous 
simplifications.  Many  of  the  terms  used  in  DoDD  5000.1  are  vague  and  context 
related,  making  them  challenging  to  measure.  Chapter  III,  Section  G  of  this  thesis 
details  how  the  variables  of  manpower,  HFE,  habitability,  survivability,  safety, 
and  performance  were  quantified  for  the  purposes  of  this  thesis  effort. 
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II.  LITERATURE  REVIEW 


A.  OVERVIEW 

This  chapter  provides  a  general  overview  of  the  HSI  literature.  Sections  B 
and  C  discuss  the  chain  of  guidance,  in  the  order  of  precedence,  that  applies  to 
the  United  States  Navy.  This  chain  of  guidance  represents  just  one  of  many 
document  chains  branching  off  the  DoD  HSI  guidance.  Section  D  discusses  and 
summarizes  two  books  on  HSI.  Section  E  concludes  the  HSI  literature  review 
with  a  summary  of  how  all  the  sections  build  a  picture  of  the  current  state  of  HSI 
within  the  DoD.  Section  F  highlights  the  relationships  among  the  HSI  domains, 
with  a  focus  on  safety.  Section  G  introduces  NPS’s  concept  of  HSI.  Section  H 
describes  the  two  of  the  three  data  sources  used  in  this  thesis:  the  official  U.S. 
Navy  mishap  reports  and  Status  of  Resources  and  Training  System  (SORTS) 
reports. 

B.  DEPARTMENT  OF  DEFENSE  (DoD)  HSI  GUIDANCE 

DoDD  5000.1,  The  Defense  Acquisition  System,  and  DoDI  5000.2, 
Operation  of  the  Defense  Acquisition  System,  reference  each  other  and  together 
provide  the  superseding  guidance  for  all  DoD  acquisition  programs.  The  Defense 
Acquisition  Guidebook  (DAG)  contains  suggestions  and  recommended  best 
practices  for  implementing  DoDD  5000.1  and  DoDI  5000.2  (DAU  Website,  2004). 
The  Chairman,  Joint  Chiefs  of  Staff  Instruction  (CJCSI)  31 70.01  F  and  the 
Chairman,  Joint  Chiefs  of  Staff  Manual  (CJCSM)  31 70.01  C  provide  guidance  for 
Joint  Capabilities  Integration  and  Development  System  (JCIDS)  acquisition 
programs  (CJCSI  3170.01  F,  2007,  CJCSM  31 70.01  C,  2007). 

1.  Department  of  Defense  Directive  (DoDD)  5000.1 

DoDD  5000.1,  Section  1.2  states,  “This  directive,  along  with  reference  (b) 
[DoDI  5000.2],  provides  management  principles  and  mandatory  policies  and 
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procedures  for  managing  all  acquisition  programs”  (DoDD  5000.1,  2003,  p.  2). 
Section  2,  Applicability  and  Scope,  states  that  the  DoDD  5000.1  applies  to  all 
acquisition  programs  within  the  Defense  Department. 

The  only  direct  reference  to  HSI  is  Enclosure  1.29,  Total  Systems 
Approach,  which  directs  program  managers  to  “apply  human  systems  integration 
to  optimize  total  system  performance  (hardware,  software,  and  human), 
operational  effectiveness,  and  suitability,  survivability,  safety,  and  affordability” 
(DoDD  5000.1, 2003,  p.  El -29).  Enclosure  1.29  goes  on  to  stress  total  ownership 
and  life-cycle  considerations.  When  viewed  online,  the  words  “human  system 
integration”  hyperlink  to  Chapter  6  of  the  DAG. 

2.  Department  of  Defense  Instruction  (DoDI)  5000.2 

The  DoDI  5000.2  purpose  section  states,  “This  instruction:  implements 
reference  (c)  [DoDD  5000.1],  the  guidelines  of  references  (d)  and  (e),  and  current 
laws”  (DoDI  5000.2,  2003,  p.  2).  DoDI  5000.2  has  the  same  applicability  as 
DoDD  5000.1,  to  all  DoD  acquisition  programs,  with  the  additional  guidance  that 
some  requirements  only  apply  to  Major  Defense  Acquisition  Programs  (MDAPs) 
and  Major  Automated  Information  System  (MAIS)  programs  (DoDI  5000.2,  2003). 

Enclosure  7,  Human  Systems  Integration  (HSI),  directs  the  program 
manager  to 

.  .  .  have  a  comprehensive  plan  for  HSI  in  place  early  in  the 
acquisition  process  to  optimize  total  system  performance,  minimize 
total  ownership  costs,  and  ensure  that  the  system  is  built  to 
accommodate  the  characteristics  of  the  user  population  that  will 
operate,  maintain,  and  support  the  system.  (DoDI  5000.2,  2003, 
p.  E7) 

The  enclosure  then  lists  seven  HSI  domains,  with  direction  on  how  the 
program  managers  are  to  address  each  one.  DoDI  5000.2,  Enclosure  7,  is 
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included  as  Appendix  B  of  this  thesis.  Although  it  implies  the  need  to  make 
trade-offs,  DoDI  5000.2  does  not  direct  how  to  “optimize  performance”  to 
“minimize  total  ownership  costs.” 

In  addition  to  Enclosure  7,  Section  3. 7. 1.1  mentions  HSI  as  one  of  a 
dozen  considerations  during  the  system  development  and  demonstration  phase. 
Section  3. 9. 2. 2,  under  Sustainment,  does  not  mention  HSI  by  name,  but  does 
direct  program  managers  to  consider  the  HSI  domains  of  manpower,  personnel, 
training,  HFE,  habitability,  environmental,  safety,  and  occupational  health 
(ESOH),  and  survivability. 

3.  Defense  Acquisition  Guidebook  (DAG) 

The  DAG  is  an  online  document 

designed  to  complement  [the  DoDD  5000.1  and  the 
DoDI  5000.2]  by  providing  the  acquisition  workforce  with 
discretionary  best  practice  that  should  be  tailored  to  the  needs  of 
each  program  ....  All  chapters  contain  non-mandatory  staff 
expectations  for  satisfying  the  mandatory  requirements  in 
DoD  Instruction  5000.2.  (DAG,  2004,  ^  foreword) 


Chapter  6  of  the  DAG  is  the  primary  source  of  high-level  guidance  on  HSI. 
The  overview  states,  “DoD  acquisition  policy  [DoDD  5000.1]  requires  optimizing 
total  system  performance  and  minimizing  the  cost  of  ownership  through  a  ‘total 
system  approach’  to  acquisition  management”  (DAG,  2004,  ^  6.0).  Section  6.0.1, 
Purpose,  defines  HSI  as  an  element  of  the  systems  engineering  process,  whose 
understanding  will  allow  Program  Managers  to  effectively  and  affordably 
integrate  human  capabilities  and  limitations  in  system  design  and  development 
(DAG,  2004). 

Chapter  6  has  six  subsections.  Section  6.1,  Total  System  Approach, 
briefly  reviews  DoDD  5000.1  directed  total  system  approach.  It  highlights  the 
need  to  consider  maintenance,  support,  and  training  as  parts  of  the  total  system. 
Section  6.2  has  seven  subsections,  each  one  describing  an  HSI  domain. 
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Section  6.3  provides  planning  details  for  the  HFE  domain.  Section  6.4  provides 
advice  on  HSI  implementation.  Section  6.5  describes  the  human  considerations 
associated  with  resource  estimating  and  planning.  The  last  section  of  Chapter  6, 
Section  6.6,  provides  two  reference  listings  for  additional  information 
(DAG,  2004). 

Section  6.4  articulates  a  vision  of  what  HSI  is  and,  in  general,  how  to 
conduct  it. 

The  key  to  a  successful  HSI  strategy  is  integration.  To  optimize 
total  system  performance  and  determine  the  most  effective, 
efficient,  and  affordable  design  entails  trade  studies  both  within  the 
HSI  elements  and  between  the  HSI  elements  and  the  system 
platform.  (DAG,  2004,  H  6.4) 


It  goes  on  to  recommend  that  Program  Managers  integrate  HSI 
parameters/requirements  in  the  Initial  Capabilities  Document  (ICD),  Capability 
Development  Document  (CDD),  and  Capability  Production  Document  (CPD).  In 
addition  to  Chapter  6,  recommendations  to  apply  HSI  appear  throughout  the 
DAG,  from  initial  capabilities  through  final  test  and  evaluation  to  life-cycle 
logistics. 

4.  Chairman,  Joint  Chiefs  of  Staff  Instruction  (CJCSI) 
3170.01  Series 

The  CJCSI  31 70.01  F,  Joint  Capabilities  Integration  and  Development 
System,  is  a  59-page  document  that  establishes  the  policies  and  procedures  of 
the  JCIDS  (CJCSI  3170.01  F,  2007).  The  instruction  mentions  HSI  only  five 
times — twice  in  Enclosure  B,  Policy,  and  three  times  in  the  Glossary.  Enclosure 
B  states: 

DOTMLPF  [doctrine,  organization,  training,  materiel,  leadership  and 
education,  personnel,  and  facilities]  includes  analysis  of  the  entire 
life  cycle,  including  the  sustainment;  environment,  safety,  and 
occupational  health  (ESOH);  and  all  human  systems  integration 
(HSI)  domains.  (CJCSI  3170.01  F,  2007,  p.  B-1) 
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Later,  Enclosure  B  mentions  HSI  as  one  consideration  when  identifying 
capability  gaps  and  potential  materiel  and  nonmaterial  solutions  (CJCSI 
31 70.01  F,  2007).  The  Glossary  definition  of  HSI  references  DoDI  5000.2  and 
adds  that  it  includes  the  integrated  and  comprehensive  analysis,  design,  and 
assessment  of  requirements,  concepts,  and  resources  for  system  manpower, 
personnel,  training,  safety  and  occupational  health,  habitability,  personnel 
survivability,  and  HFE.  The  Glossary  definition  of  “nonmaterial  solutions”  refers  to 
HSI  under  policy  changes.  The  Glossary  also  lists  HSI  as  a  consideration  for 
“supportability”  (CJCSI  31 70.01  F,  2007). 

5.  Chairman,  Joint  Chiefs  of  Staff  Manual  (CJCSM)  3170.01  Series 

CJCSM  3170.01  C,  Operation  of  the  Joint  Capabilities  Integration  and 
Development  System,  is  a  149-page  document  that 

.  .  .  sets  forth  guidelines  and  procedures  for  operation  of  the  Joint 
Capabilities  Integration  and  Development  System  (JCIDS) 
regarding  the  development  and  staffing  of  JCIDS  documents  in 
support  of  reference  (a)  [CJCSI  3170.01  series],  (CJCSM 
3170.01. C,  2007,  p.  1) 

Appendix  A  to  Enclosure  E,  Capability  Development  Document  Format, 
and  Appendix  A  to  Enclosure  F,  Capability  Production  Document  Format,  each 
mention  HSI  twice.  Under  “Other  DOTMLPF  and  Policy  Considerations,”  it 
advises  discussion  of  HSI  considerations  that  have  a  major  impact  on  system 
effectiveness,  suitability,  and  affordability.  Under  “Other  System  Attributes,”  it 
mentions  HSI  as  one  factor  to  “consider  as  appropriate”  (CJCSM 
3170.01C,  2007). 

6.  Conclusions  on  Department  of  Defense  (DoD)  Guidance 

The  DoD  instructions  are  high-level  and  broad.  While  the  DAG  provides 
excellent  advice,  it  is  nonbinding.  The  CJCSI  and  CJCSM  mention  HSI  as  a 
consideration,  but  do  not  define  it  or  provide  direction  about  how  to 
employ  it. 
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C.  UNITED  STATES  NAVY  HUMAN  SYSTEMS  INTEGRATION 

(HSI)  GUIDANCE 

This  section  looks  at  the  chain  of  instructions  that  pertain  to  the 
implementation  of  HSI  for  U.S.  Navy  systems.  These  instructions  provide 
clarification  and  further  guidance  to  the  DoD  and  CJCS  guidance  already 
discussed.  This  section  presents  the  instructions  by  order  of  precedence,  from 
highest  to  lowest.  At  each  level,  the  documents  provide  requirements  in  addition 
to  the  requirements  in  the  documents  they  reference. 

1.  Secretary  of  the  Navy  (SECNAV)  Instruction  5000.2  Series 

SECNAV  Instruction  5000. 2D,  Implementation  and  Operation  of  the 
Defense  Acquisition  System  and  the  Joint  Capabilities  Integration  and 
Development  System,  is  a  195-page  instruction  that  includes  guidance  for  HSI.  It 
issues  requirements  for  major  and  nonmajor  defense  acquisition  and  information 
technology  programs  for  the  Department  of  the  Navy  (DON)  (SECNAV  5000. 2D, 
2008).  It  also  cites  the  DAG  as  a  discretionary  companion  to  the  instruction. 

Section  1.7.k,  under  Responsibilities,  identifies  the  Director,  Total  Force 
Requirements  Division,  (CNO  [N 1 2]),  as  the  single  governing  authority  for  HSI 
policy,  requirements,  and  resources.  The  N12  is  directed  to  participate  in  the 
identification  of  enterprise  Manpower,  Personnel,  Training  (MPT)  and  education 
shortfalls,  and  to  investigate  innovative  approaches  and  solutions  to  optimize 
manpower  and  improve  performance  (SECNAV  5000. 2D,  2008). 

The  primary  HSI  guidance  in  SECNAV  5000. 2D  is  in  Enclosure  7, 
Systems  Engineering  and  Human  Systems  Integration.  It  directs  program 
managers  to  include  HSI  in  a  systems  engineering  process  used  to  translate 
operational  requirements/capability  needs  into  a  system  solution  (SECNAV 
5000. 2D,  2008).  Section  7. 1.6. 2,  explicitly  directs  information  technology  (IT) 
systems  to  address  interoperability  and  HSI  considerations  (SECNAV  5000. 2D, 
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2008).  Section  7.2.1,  HSI  in  Acquisition,  specifically  addresses  HSI  and  defines 
the  scope  of  acquisition  programs,  to  include  modifications  to  legacy  systems 
(SECNAV  5000. 2D,  2008). 

Sections  7.2.2  through  7.2.5  of  SECNAV  5000. 2D  provide  one  paragraph 
each  on  how  to  consider  the  HSI  domains.  Section  7.2.2  lists  MPT  as  a  single 
factor.  The  other  three  sections  are  7.2.3,  HFE;  7.2.4,  Personnel  Survivability; 
and  7.2.5,  Habitability.  Section  7.3  addresses  Environmental,  Safety,  and 
Occupational  Health  (ESOH)  considerations  and  is  seven  pages  long,  including 
two  full-page  tables  (SECNAV  5000.2D,  2008). 

HSI  is  briefly  mentioned  in  Section  2. 1.2. 3  under  the  discussion  of 
manpower  requirements. 

DOTMLPF  analyses,  conducted  as  the  first  step  in  the  Functional 

Solutions  Analysis,  shall  address  all  eight  Human  Systems 

Integration  (HSI)  domains  including  Environmental,  Safety,  and 

Occupational  Health  (ESOH).  (SECNAV  5000.2D,  2008,  p.  E2-10) 

Section  3.4  is  Acquisition  Strategy.  Section  3.4.7. 1,  Human  Systems 
Integration  (HSI),  mandates  an  acquisition  strategy  that  summarizes  HSI 
planning,  including  how  the  program  will  meet  HSI  program  requirements  and 
standards  (SECNAV  5000.2D,  2008). 

2.  Chief  of  Naval  Operations  Instruction  5310.23  Series 

OPNAVINST  5310.23  is  currently  in  draft  format.  If  signed,  it  will  reference 
SECNAVINST  5000. 2D  and  will  replace  reference  (a),  Chief  of  Naval  Operations 
Guidance  for  2004,  of  the  Naval  Sea  Systems  Command  (NAVSEA)  Instruction 
3900.8  Series. 

3.  Naval  Sea  Systems  Command  (NAVSEA)  Instruction 
3900.8  Series 

NAVSEA  Instruction  3900. 8A  is  Human  System  Integration  (HSI)  Policy  in 

Acquisition  and  Modernization  Programs  (NAVSEA  3900. 8A,  2005).  This 
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16-page  document  expands  the  definition  of  acquisition  programs  to  include 
research  and  development,  modernization,  and  disposal  programs. 

Section  6  of  NAVSEA  3900. 8A,  Responsibilities,  outlines  the  roles  and 
tasks  for  each  level  from  the  NAVSEA  03  to  Ship  Design  Managers.  Section  6 
directs  NAVSEA  Deputy  Commanders  to  “ensure  manpower  and  training 
trade-offs  studies  are  conducted  and  documented  ...  to  minimize  manning  and 
training  costs”  (NAVSEA  3900. 8A,  2005,  p.  4). 

Enclosure  1  provides  the  first  detailed  guidance  on  how  to  develop  and 
implement  an  HSI  plan.  Enclosure  1  explicitly  states  that  an  interdependent 
relationship  exists  between  the  HSI  domains,  and  trade-offs  made  in  a  program 
can  have  an  impact  across  all  of  the  domains.  It  mandates  the  development  of 
HSI  criteria  for  all  programs.  Enclosure  1  cites  20  instructions  and  industry 
standards  as  references  to  assist  in  developing  and  evaluating  HSI  trade-offs. 

4.  Conclusions  on  United  States  Navy  Human  Systems 
Integration  (HSI)  Guidance 

The  Navy’s  HSI  guidance  expands  the  scope  to  include  modernization 
programs  of  legacy  systems.  Of  all  the  DoD  and  U.S.  Navy  instructions  reviewed, 
NAVSEA  3900. 8A  is  the  only  one  that  mentions  trade-offs  across  all  domains. 

D.  ACADEMIC  HUMAN  SYSTEMS  INTEGRATION  (HSI)  GUIDANCE 

1.  Handbook  of  Human  Systems  Integration  (HSI) 

The  Handbook  of  Human  Systems  Integration  (Booher,  2003)  is  a 
965-page  tome  containing  the  most  extensive  guidance  available  on  HSI.  It  is  the 
collected  work  of  approximately  90  contributors,  technical  advisors,  and 
reviewers  from  the  government,  industry,  and  academia.  Contributors  reported 
the  status  of  HSI  in  the  United  Kingdom,  Canada,  the  United  States  Army,  the 
United  States  Navy,  the  United  States  Air  Force,  the  Federal  Aviation 
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Administration,  and  the  National  Academy  of  Sciences.  Many  of  the  chapters 
cover  both  military  and  nonmilitary  applications  (Booher,  2003). 

Section  1.2.1  defines  HSI  as  primarily  a  technical  and  managerial  concept, 
with  specific  emphasis  on  methods  and  technologies  that  can  be  utilized  to  apply 
the  HSI  concept  to  systems  integration  (Booher,  2003).  This  circular  definition  is 
followed  by  a  section  that  quotes  the  U.S.  Army’s  Manpower  and  Personnel 
Integration  (MANPRINT)  and  DoDI  5000.2  definitions  of  the  “HSI  concept.”  The 
preface  describes  HSI  as  a  very  attractive  new  integrating  discipline  that  can  help 
move  business  and  engineering  cultures  toward  a  people-technology  orientation 
(Booher,  2003). 

In  the  HSI  Handbook,  Booher  describes  HSI  as  an  addition  to  the  systems 
engineering  (SE)  process.  The  book  cautions, 

If  organizations  are  to  change  significantly  to  take  full  advantage  of 
the  benefits  that  HSI  can  offer,  I  [Booher]  believe  this  is  most  likely 
to  be  accomplished  as  an  inherent  part  of  systems  engineering  and 
management.  (Booher,  2003,  p.  xvi) 


Though  HSI  developed  from  previous  HFE  and  SE  work,  it  is  more  than 
just  HFE  or  “good”  systems  engineering.  Booher  states  in  the  foreword: 

Human  factors  and  ergonomics,  while  important  components 
are  not  themselves  sufficient  because  they  do  not  cover  other 
important  human  domains  ....  The  major  component  currently 
missing  from  systems  engineering  and  management  is  a  detailed 
description  of  the  principles  and  methods  of  human  systems.  The 
intent  of  the  Handbook  is  to  provide  that  component.  (Booher, 

2003,  p.  xvi) 

Acknowledging  the  broad  scope  of  HSI  and  the  diversity  of  contributors, 
the  handbook  is  designed  to  help  users  identify  the  sections  of  most  interest  to 
them.  The  chapters  are  grouped  into  sections  and  each  section  and  chapter  start 
with  an  overview  (Booher,  2003).  The  chapters  that  are  most  relevant  to  this 
thesis  are  Chapters  8  and  17,  which  address  interactions  and  trade-offs  between 
all  of  the  domains. 
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Chapter  8,  written  by  Barnes  and  Beevis,  starts  by  citing  the  DoD  direction 
for  HSI  in  military  systems.  It  discusses  human  performance  measurement 
issues  related  to  system  design,  with  a  focus  on  those  issues  that  require 
trade-offs.  When  discussing  the  development  process  Barnes  and  Beevis  state, 

Since  a  single  model  or  simulation  that  is  in  any  sense  complete  is 
usually  not  available  early  in  the  design  process,  the  HSI 
practitioner  must  rely  on  a  combination  of  modeling,  human 
experimentation,  and  simulation  for  each  stage  of  development. 
(Booher,  2003,  p.  237) 

In  Section  8.7,  Interactions  Among  HSI  Domains,  Barnes  and  Beevis  add 
that  when  planning  human  performance  measures  for  a  system  under 
development,  it  is  important  to  remember  that  the  various  HSI  domains  are 
interrelated  and  that  changes  in  design  to  improve  one  domain  nearly  always 
affect  other  domains  (Booher,  2003).  Supporting  the  present  study  they  note  that, 
“Although  the  need  is  obviously  great,  there  is  no  well-established  body  of 
knowledge  on  HSI  domain  trade-offs”  (Booher,  2003,  p.  254). 

Chapter  17,  by  Rouse  and  Boff,  addresses  types  of  HSI  investments  that 
have  intangible  characteristics  in  addition  to  more  tangible  attributes.  For 
example,  investing  in  more  training  or  higher  personnel  selection  standards  can 
result  in  less  health  and  safety  costs  and  higher  morale  and  productivity.  Rouse 
and  Boff  also  note  the  difficulty  in  balancing  trade-offs  between  short-term  and 
long-term  benefits  (Booher,  2003). 

Chapter  17  recommends  using  a  multiattribute  utility  theory  (MAUT)  model 
to  allow  each  stakeholder  to  assign  “costs”  and  “benefit”  values  to  each  system 
attribute  and  time.  This  allows  for  quantitative  weights  of  qualitative  things, 
making  assumptions  explicit  and  open  to  testing.  Rouse  and  Boff  found  that  use 
of  the  MAUT  methodology  documents  underlying  assumptions,  making  them 
explicit  and  open  to  review  (Booher,  2003).  Quantifying  and  comparing  utility 
inputs  can  be  difficult  with  large  numbers  of  variables.  Evaluating  costs  and 
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benefits  is  further  complicated  by  investment  costs  spread  over  time,  uncertain 
returns,  multiple  stakeholders  with  different  values,  and  indirect  and 
intangible  returns. 

2.  Human-System  Integration  (HSI)  in  the  System  Development 
Process:  A  New  Look 

The  United  States  Army  and  United  States  Air  Force  funded  this  384-page 
report  by  the  National  Academy  of  Science’s  Committee  on  Human-System 
Design  Support  for  Changing  Technology.  The  introduction  defines  HSI  as  a 
discipline  concerned  with  ensuring  that  the  characteristics  of  people  are 
considered  throughout  the  system  development  process,  with  regard  to  their 
selection  and  training,  their  participation  in  system  operation,  and  their  health  and 
safety  (Pew  &  Mavor,  2007). 

The  authors  propose  a  “risk-driven”  approach  that  achieves  cost 
effectiveness  by  focusing  resources  on  high-risk  aspects  of  a  program,  while 
deemphasizing  resources  spent  on  low-risk  aspects  (Pew  &  Mavor,  2007).  The 
focus  of  this  book  is  similar  to  the  focus  of  Chapter  17  of  the  Handbook  of  Human 
Systems  Integration.  This  book  differs  from  Chapter  17  of  the  Handbook  of  HSI 
with  respect  to  benefit  considerations.  Pew  and  Mavor  contend  that  HSI  risk 
trade-offs  must  be  presented  in  terms  of  acquisition  and  life-cycle  costs  and 
benefits,  as  opposed  to  including  intangible  benefits  (Pew  &  Mavor,  2007).  They 
agree  with  Chapter  17  of  the  Handbook  of  HSI  on  the  need  for  a  “shared 
representation”  that  can  make  assumptions  explicit  and  open  to  review  (Pew  & 
Mavor,  2007). 

In  addition  to  supporting  the  present  work  in  trade  space  development, 
this  book  reinforced  the  past  choices  and  influenced  the  current  choices  of 
variables.  In  the  conclusions  to  one  of  their  examples,  they  noted  that  HFE 
concerns  are  often  linked  to  MPT  concerns  (Pew  &  Mavor,  2007).  This  supports 
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the  first  design  of  the  NPS  trade  space  by  Simpson  (2006),  with  MPT  and  HFE 
as  inputs  that  determine  the  levels  of  performance,  risk,  and  the  other 
HSI  domains. 


The  book  defines  two  different  types  of  risk:  operational  risk  and  program 
risk.  Figure  1  outlines  the  types  of  risk. 


Figure  1 .  Taxonomy  of  Risk 


The  book  defines  operational  risk  as  safety  and  usability  risk.  Safety  is  the 
risk  that  users  will  cause  damage  or  be  injured  while  using  the  item.  Usability  risk 
is  the  risk  that  operators  will  not  use  the  item  fully  or  at  all.  Program  management 
risk  is  the  risk  of  exceeding  budgeted  cost  and  the  risk  of  exceeding  allotted 
schedule  (Pew  &  Mavor,  2007). 

E.  HUMAN  SYSTEMS  INTEGRATION  (HSI)  CONCLUSIONS 

One  obstacle  to  effective  HSI  in  DoD  programs  is  the  long-term  and 
uncertain  nature  of  life-cycle  costs.  While  the  literature  suggests  that  HSI  should 
consider  operational  risks  and  life-cycle  costs,  current  DoD  and  Navy  instructions 
only  apply  to  the  acquisition  process,  which  evaluates  performance  based 
predominately  on  short-term  program  risk.  The  other  main  obstacle  is  that, 
despite  discussions  of  the  interconnections  between  the  domains,  the  HSI 
literature  focuses  primarily  on  the  individual  domains  of  HSI.  DoD  instructions 
discuss  HFE  and  ESOH  separately  from  the  other  domains,  while  MPT  tend  to 
be  grouped  together. 
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Several  factors  propagate  this  condition.  All  of  the  HSI  domains,  with  the 
exception  of  system  survivability,  are  stand-alone  fields  that  predate  HSI.  Most 
HSI  practitioners  have  education  and  background  in  only  one  or  two  of  the  HSI 
domains.  In  DoD  and  private  industry,  HSI  domains  are  often  separate  functional 
areas  in  different  parts  of  the  organizations.  All  of  these  factors  highlight  the  need 
for  “shared  representations”  that  can  take  inputs  from  experts  in  one  domain,  and 
display  and  translate  them  into  effects  on  the  other  domains  and  on  total 
system  performance. 

F.  SAFETY  LITERATURE 

Safety  considerations  have  traditionally  been  one  of  the  largest  drivers  in 
HSI  trade-offs.  As  seen  in  the  DoD  and  U.S.  Navy  instructions  reviewed 
previously,  safety  is  often  discussed  separately  and  often  in  greater  detail  than 
the  other  HSI  domains.  This  is  not  surprising,  as  it  was  the  cost  of  airplane 
mishaps  during  World  War  II  that  formalized  HFE,  a  primary  domain  of  HSI,  into 
a  discipline.  Therefore,  an  evaluation  of  safety  literature  from  the  United  States 
Navy  and  civilian  industries  is  relevant  to  this  thesis. 

A  2004  study  by  the  Marine  Accident  Investigation  Branch  (MAIB)  of  the 
United  Kingdom  looked  at  66  mishaps  of  merchant  vessels  operating  near  the 
coast  of  the  British  Isles.  The  study  determined  that  having  only  two  watch 
standers  taking  turns  led  to  fatigue  and  inadequate  performance.  It 
recommended  adding  a  third  person  to  the  rotation  and  having  a  lookout  in  all  but 
clear  weather  with  light  traffic  (MAIB,  2004).  The  larger  watch  team  size  of  Navy 
ships  does  not  prevent  similar  mishaps.  In  an  opinion  piece  from  Fathom 
magazine,  Rear  Admiral  Finney  discussed  two  almost  identical  collisions,  each 
between  a  Navy  destroyer  (DD)  and  a  merchant  ship,  in  the  same  shipping 
channel,  30  months  apart.  He  notes  that,  in  both  cases,  the  crew  failed  to  follow 
established  procedures,  failed  to  communicate,  and  lookouts  were  missing 
required  equipment  (Finney,  1992). 
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Thus,  while  manning  levels  and  personnel  are  factors  in  mishap  rates,  as 
the  size  of  the  crew  increases,  manning  and  personnel  factors  have  less 
influence  than  factors  such  as  team  performance  and  workload.  An  investigation 
of  manning  onboard  18  U.S.  Navy  ships  by  the  Naval  Health  Research  Center 
found  that  manpower  utilization  had  a  stronger  relationship  with  performance  and 
illness  levels  than  did  the  manning  level  (Dean,  Harvey,  Pugh,  & 
Gunderson,  1979). 

As  another  example,  research  efforts  by  National  Aeronautical  and  Space 
Agency  (NASA)  and  the  Federal  Aviation  Administration  (FAA)  in  the  late  1970s 
found  that  human  error  and  teamwork  failures,  and  not  mechanical  malfunction, 
were  the  major  causes  of  industrial  accidents.  This  led  to  the  development  of 
Crew  Resource  Management  (CRM)  training  programs  by  commercial  airlines. 
CRM  programs  teach  nontechnical  skills,  such  as  teamwork  and  communication, 
to  improve  operational  performance.  Training  programs  include  both  initial 
training  and  regular  refresher  training.  In  addition  to  its  current  use  by  all  major 
international  airlines,  the  CRM  concept  is  applied  in  military  aviation, 
anesthesiology,  air  traffic  control,  the  Merchant  Navy,  the  nuclear  power  industry, 
aviation  maintenance,  and  the  offshore  oil  industry  (Flin,  O’Connor,  & 
Mearns,  2002). 

G.  NPS  HUMAN  SYSTEMS  INTEGRATION  (HSI)  CONCEPT 

This  thesis  builds  on  a  theoretical  framework  proposed  in  an  NPS  thesis, 
“Toward  an  Improved  Method  of  HSI  Evaluation  in  Defense  Acquisition” 
(Simpson,  2006).  The  previous  thesis  proposed  a  model  for  an  HSI  trade  space 
tool  (TST)  with  four  input  variables  and  four  output  measures.  Manpower, 
personnel,  training,  and  HFE  are  the  four  input  variables;  the  remaining  HSI 
domains  are  functions  of  the  interactions  of  the  inputs  and  the  output  is  total 
system  performance. 

The  model  represented  in  Figure  2  looks  at  how  total  system  performance 
is  determined  by  inputs  of  MPT  and  HFE  with  enabling/constraining  factors  of 
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cost,  schedule,  and  risk.  The  final  outcome,  total  system  performance,  is  the 
combination  of  all  of  the  first  order  outcomes,  which  includes  mishap  rates 
(Miller  &  Shattuck,  2007). 
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©  Miller  and  Shattuck,  2007 

NPS  HSI  Process  Model  (From  Miller  &  Shattuck,  2007) 


From  this  model,  a  tradespace  tool  (TST)  was  developed  by  the  NPS 
Office  of  Continuous  Learning  (OCL).  The  initial  version  of  the  TST  was 
developed  during  thesis  efforts  by  Simpson  (2006).  Figures  3  and  4  show  the 
input  screens  for  the  NPS  HSI  TST. 
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Figure  3.  NPS  HSI  TST  Input  Screen 


The  user  defined  the  relationships  between  four  variables  with  each  other, 
and  with  the  acquisition  variables  of  cost,  schedule,  performance,  and  risk.  Once 
these  22  interactions  were  defined,  the  user  progressed  to  a  second  screen, 
shown  in  Figure  4.  This  screen  contained  eight  slider  bars,  four  for  the  input 
variables  and  four  for  the  output  variables.  Moving  any  one  of  the  input  slider 
bars  shifted  all  four  of  the  output  bars  according  to  the  defined  relationships. 


20 


H.  NAVY  REPORT  FORMATS 

This  section  describes  the  format  for  the  two  data  sets  collected,  U.S. 
Naval  Safety  Center  Mishap  Reports  and  SORTS  reports. 

1.  Naval  Safety  Center  Mishap  Reports 

The  Chief  of  Naval  Operations  Instruction  (OPNAVINST)  5102.1  D  (2005) 
defines  Class  A,  B,  and  C  mishaps  in  this  way: 


1 .  Class  A  Mishap.  The  resulting  total  cost  of  damages  to  DoD  or 
non-DoD  property  in  an  amount  of  $1  million  or  more;  a  DoD 
aircraft  is  destroyed;  or  an  injury  and/or  occupational  illness  result 
in  a  fatality  or  permanent  total  disability. 

2.  Class  B  Mishap.  The  resulting  total  cost  of  damages  to  DoD  or 
non-DoD  property  is  $200,000  or  more,  but  less  than  $1  million.  An 
injury  and/or  occupational  illness  result  in  permanent  partial 
disability  or  when  three  or  more  personnel  are  hospitalized  for 
inpatient  care  (beyond  observation)  as  a  result  of  a  single  mishap. 
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3.  Class  C  Mishap.  The  resulting  total  cost  of  damages  to  DoD  or 
non-DoD  property  is  $20,000  or  more,  but  less  than  $200,000;  a 
nonfatal  injury  that  causes  any  loss  of  time  from  work  beyond  the 
day  or  shift  on  which  it  occurred;  or  a  nonfatal  occupational  illness 
that  causes  loss  of  time  from  work  or  disability  at  any  time 
(OPNAVINST  5102.1  D,  2005,  p.  2-1). 

OPNAVINST  51 02. ID  requires  investigation  and  reporting  of  all  Class  A, 
B,  and  C  mishaps.  It  does  not  define  a  Class  D  mishap.  Based  on  the  definitions 
listed  above,  a  Class  D  mishap  could  be  defined  as  an  incident  that  resulted  in 
less  than  $20,000  of  damages  or  in  a  nonfatal  injury  that  did  not  cause  any  loss 
of  time  from  work  beyond  the  day  or  shift  on  which  it  occurred.  The  OPNAVINST 
also  directs  that  “near  mishaps  should  be  investigated  and  reported  via  a 
HAZREP  [Navy  message]”  (OPNAVINST  5102.1  D,  2005,  p.  1-12).  The  mishap 
reporting  matrix  specifically  directs  a  hazard  report  for  cases  of  “Electrical  shock 
afloat  from  a  design  deficiency  (no  injury)”  and  “Man-overboard  mishap 
underway  with  no  injury”  (OPNAVINST  51 02. ID,  2005,  p.  5-8).  The  OPNAVINST 
provides  discretion  to  unit  commanding  officers  on  whether  or  not  to  report 
Class  D  mishaps  and  near  misses. 

2.  Status  of  Resources  and  Training  System  (SORTS)  Reports 

Each  U.S.  Navy  ship  submits  a  monthly  SORTS  report.  It  contains  five 
columns  for  reported  status:  overall  status,  personnel  status,  equipment  condition 
status,  supply/equipment  status,  and  training  status.  Each  reported  status  ranges 
from  best,  (Category  One  or  C-1),  to  worst,  (Category  Five  or  C-5).  The  following 
excerpt  from  Commander,  Military  Sealift  Command  Instruction  (COMMSCINST) 
3501 ,2B  (1998)  defines  each  of  the  levels  for  the  overall  category. 

5.4.1  Category  Levels  Defined 

The  overall  unit  category  level  will  be  based  only  upon  organic 
resources  and  training  under  the  operational  control  of  the  reporting 
unit  or  its  parent  unit.  The  five  categories  of  overall  unit  category 
levels  are: 
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a.  C-1.  Unit  possesses  the  required  resources  and  is  trained  to 
undertake  the  full  wartime  mission  for  which  it  is  organized  or 
designed. 

b.  C-2.  Unit  possesses  the  resources  and  has  accomplished  the 
training  necessary  to  undertake  the  bulk  of  wartime  mission  for 
which  it  is  organized  or  designed. 

c.  C-3.  Unit  possesses  the  resources  and  has  accomplished  the 
training  necessary  to  undertake  major  portions  of  the  wartime 
mission  for  which  it  is  organized  or  designed. 

d.  C-4.  Unit  requires  additional  resources  and/or  training  to 
undertake  its  wartime  mission,  but  if  the  situation  dictates,  it  may  be 
directed  to  undertake  portions  of  its  wartime  mission  with  resources 
on  hand. 

e.  C-5.  Unit  is  undergoing  a  service-directed  resource  change  and 
is  not  prepared,  at  this  time,  to  undertake  the  wartime  mission  for 
which  it  is  organized  or  designed.  C-5  units  are  restricted  to: 

(1 )  Ship  in  overhaul  or  restricted  availability  for  30  days  or  longer. 

(2)  Units  undergoing  major  equipment  conversion/  transition. 

(3)  Units  placed  in  cadre  status  by  the  parent  service. 

(4)  Units  being  activated,  inactivated  and  reactivated. 

(5)  Units  not  manned  or  equipped  but  required  in  the  wartime  force 
structure. 

(6)  Units  tasked  as  training  units  that  could  be  tasked  to  perform  a 
wartime  mission. 

5.4.2  Category  Levels,  Caveats 

The  overall  unit  category  level  will  be  identical  to  the  lowest  level 
[highest  number]  recorded  in  any  of  the  unit's  individually  measured 
resource  areas  of  personnel,  equipment  and  supplies  on  hand, 
equipment  condition  and  training,  unless  subjectively  raised  or 
lowered  by  the  unit  commander.  However,  modification  of  a  unit's 
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overall  level  by  its  commander  does  not  permit  modification  of  the 
computed  status  of  each  individually  measured  area,  which  must 
be  reported  without  adjustment.  (COMSCINST  3501. 2B,  1988, 
pp.  15-16) 

All  U.S.  Navy  ships  undergo  overhaul  and  maintenance  from  time  to  time.  We 
wanted  to  focus  the  model  for  this  thesis  on  only  those  months  in  which  ships 
were  in  operational  status.  However,  it  was  not  possible  to  determine  exactly 
when  a  ship  was  in  a  scheduled  maintenance  period.  Based  on  the  definitions  of 
the  overall  category,  some  ship  scheduling  information  can  be  derived  from  the 
SORTS  report.  Therefore,  those  months  when  a  ship  reported  a  C-5  rating  were 
attributed  to  planned  maintenance  activities,  rather  than  lack  of  manning, 
insufficient  training,  or  equipment  malfunctions. 
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III.  APPROACH 


A.  OVERVIEW 

The  initial  version  of  the  TST,  developed  by  NPS’s  Office  of  Continuous 
Learning  (OCL)  and  based  on  the  model  proposed  by  Miller  and  Shattuck  (2007), 
had  22  user-defined  correlations.  The  four  predictor  variables  were  manpower, 
personnel,  training,  and  HFE.  The  user  defined  the  interactions  of  these  four 
inputs  with  each  other,  and  with  the  acquisition  variables  of  cost,  schedule, 
performance,  and  risk.  Since  it  was  not  possible  to  obtain  sufficient  data  on  all 
variables  of  interest,  this  thesis  used  manning,  SORTS,  and  mishap  data  to  test 
whether  there  were  relationships  among  the  variables  that  were  available. 

Section  B  identifies  the  variables  tested  in  this  thesis.  Section  C  states  the 
hypotheses  tested.  To  test  the  model,  data  were  collected  on  the  U.S.  Navy’s 
Oliver  Hazard  Perry  Class  Frigate  (FFG-7).  Section  D  provides  background 
information  on  the  FFG-7  class.  Section  E  details  the  process  used  to  collect  the 
data.  Section  F  describes  how  the  obtained  data  were  cleaned  and  prepared  for 
analysis.  Section  G  describes  how  the  data  were  used  to  quantify  each  variable. 

B.  VARIABLE  DEFINITION 

The  predictor  variable  analyzed  in  this  thesis  was  manning  level. 

The  outcome  variables  analyzed  in  this  thesis  were  performance  (SORTS) 
and  safety  (mishaps). 

Variables  for  which  insufficient  data  were  obtained  (e.g.,  personnel  and 
training)  and  those  not  applicable  to  this  system  (e.g.,  acquisition  schedule), 
were  not  investigated  in  this  thesis.  Limiting  the  data  collection  to  the  FFG-7 
class  was  an  attempt  to  hold  other  variables,  such  as  HFE,  constant.  Data 
sources  and  information  gathered  on  some  of  the  controlled  variables 
(e.g.,  manning  cost)  are  discussed  in  Chapter  VI,  under  improvements  to  HSI 
data. 
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c. 


HYPOTHESES 


The  purpose  of  this  thesis  is  to  use  historical  data  to  explore  the  nature  of 
the  relationships  that  are  embedded  in  an  existing  notional  model  of  HSI.  Only 
those  relationships  for  which  there  were  appropriate  and  sufficient  data  were 
analyzed.  The  hypothesized  relationships  are: 

•  There  is  a  negative  relationship  between  manning  and  SORTS 
levels  (i.e.,  as  manning  increases,  performance  as  measured  on 
SORTS  will  improve). 

•  There  is  a  negative  relationship  between  manning  and  mishaps 
(i.e.,  as  manning  levels  go  down,  mishaps  will  increase). 

•  There  is  a  positive  relationship  between  mishaps  and  SORTS 
levels  (i.e.,  as  mishaps  increase,  performance  as  measured  on 
SORTS  will  decrease). 

D.  USN  FRIGATE  OVERVIEW 

The  scope  of  this  thesis  depended  on  two  major  decisions.  The  first 
decision  was  which  class  of  naval  platforms  to  evaluate.  The  Navy  fact  files  at 
http://www.navv.mil/navvdata/fact.asp  provide  basic  information  on  aircraft  and 
ship  types.  As  discussed  in  Chapter  I,  this  thesis  focused  on  the  Navy  “long  hull” 
Frigate  FFG-7  class.  The  second  decision  was  the  time  period  for  the  data 
request.  The  United  States  Navy  commissioned  the  last  ship  of  the  class,  the 
USS  Ingraham,  (FFG-61),  in  August  1989;  thus,  the  initial  requests  were  for  data 
from  October  1989  to  October  2006.  This  time  period  was  later  reduced  to 
October  1994  through  September  2006,  due  to  lack  of  available  data.  The  end 
result  was  96  months  of  data  for  each  of  the  30  U.S.  Navy  Frigates  active  at  the 
time. 

Bath  Iron  Works  and  Todd  Shipyards  built  55  FFG-7  class  ships  in  two 

major  variants.  The  United  States  Navy  received  51  and  the  Australian  Navy 

received  four  of  these  ships.  The  “short”  class  was  445  feet  long.  The  “long” 
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class  was  453  feet  long  (FAS,  2000).  A  list  of  commissioning  and 
decommissioning  dates  for  all  Navy  Frigates  was  created,  based  on  information 
from  two  private  Websites:  the  “Unofficial  U.S.  Navy  Site”  at 
http://navysite.de/friqates.htm  and  “Federation  of  American  Scientists,  Military 
Analysis  Network”  at  http://www.fas.org/man/dod-101/svs/ship/ffq-7.htm.  The  list 
of  decommissioning  dates  was  compared  to  the  official  Navy  FFG  fact  sheet 
Website  at  http://www.navv.mil/navydata/fact  displav.asp?cid=4200&tid=1300& 
ct=4.  The  Navy  fact  sheet  does  not  include  decommissioning  dates,  but  the  list  of 
decommissioned  ships  corresponds  to  all  of  the  “short”  class  FFGs.  All  of  the 
“long”  class  FFGs  are  listed  as  currently  active.  The  Naval  Manpower  Analysis 
Center  (NAVMAC)  and  the  FFG  Class  Squadron  (CLASSRON)  confirmed  this 
information  and  provided  a  list  of  Unit  Identification  Codes  (UlCs)  for  each 
active  FFG. 

E.  DATA  COMPILATION  METHOD 

This  section  describes  the  process  used  to  collect  the  data.  Locations  of 
contact  information  are  described,  rather  than  using  specific  names,  phone 
numbers,  and  e-mail  addresses.  This  was  done  for  two  reasons:  military 
personnel  change  jobs  frequently  and  it  aids  the  research  performed  on  various 
Navy  platforms. 

1.  Homeports 

The  homeport  of  each  active  FFG  is  on  the  official  Navy  FFG  fact  sheet 
Website  at  http://www.navv.mil/navydata/fact  display. asp?cid=4200&tid=1300& 
ct=4www. navy. mil.  As  of  November  2008,  this  Website  was  last  updated  in 
October  2007.  The  locations  of  the  30  ships  are  discussed  in  Chapter  IV, 
Results. 
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2. 


Unit  Identification  Codes  (UlCs) 


A  listing  of  FFG  UlCs  was  necessary  to  obtain  the  manning  and  SORTS 
data.  The  FFG  Class  Squadron  (CLASSRON)  did  not  have  historical  manning  or 
SORTS  data,  but  provided  a  list  of  UlCs  for  all  active  duty  FFGs. 

The  initial  contact  with  the  FFG  CLASSRON  was  by  telephone.  The 
administrative  department  of  Commander,  Naval  Surface  Forces,  Pacific  (CNSP) 
provided  the  phone  number.  Subsequent  communications  were  conducted  via  e- 
mail.  The  CLASSRON  for  FFGs  is  Destroyer  Squadron  14  out  of  Mayport, 
Florida.  Contact  phone  numbers  are  located  on  the  bottom  of  the  “Command” 
page  of  their  homepage  at  http://www.cds14.surfor.navv.mil/default.aspx. 

To  find  contact  information  for  other  Navy  ship  classes,  the  Commander, 
Naval  Surface  Forces,  Pacific  Website  contains  a  page  of  all  Surface  Navy 
CLASSRONs  at  http://www.surfpac.navy.mil/site%20paqes/classrons.aspx.  This 
page  contains  lists  of  links  to  the  homepage  of  each  ship,  within  each 
CLASSRON.  The  CNSP  Website  does  not  have  links  to  the  CLASSRON’s 
homepages.  The  administrative  department  of  CNSP  may  have  phone  numbers 
for  different  CLASSRONs.  Commander  Navy  Surface  Forces,  Pacific  (CNSP) 
lists  their  phone  number  on  their  homepage  at 
http://www.surfpac.navy.mil/default.aspx. 

3.  Authorized  Manning  Level 

The  Naval  Manpower  Analysis  Center  (NAVMAC)  did  not  have  historic 
manning  or  SORTS  data,  but  provided  three  “Ship  Manpower  Document”  (SMD) 
reports.  The  most  recent  SMD  was  prepared  in  January  2004,  and  is  currently 
under  revision  by  NAVMAC.  The  other  SMD  reports  were  prepared  in  December 
1997  and  September  1999.  The  reports  detail  every  authorized  billet  by 
department,  division,  rank,  rate,  officer  designator,  Navy  Enlisted  Classification 


28 


(NEC)  code,  and  officer  subspecialty  code.  This  allowed  for  the  calculation  of 
percent  of  manning  compared  to  authorized  billets,  compensating  for  changes  in 
the  authorized  levels  over  time. 

The  initial  contact  with  NAVMAC  was  by  telephone.  Subsequent 
communications  were  conducted  via  e-mail.  The  Navy  Personnel  Command 
(NPC)  customer  service  line  provided  NAVMAC’s  phone  number.  NPC’s  toll-free 
customer  service  number  is  located  on  the  NPC  homepage  at 
http://www.npc.navy.mil/Channels/. 

4.  Manning  Cost 

The  Chief  of  Naval  Operations  Programming  Division,  N80,  provided  the 
Active  Duty  Programming  Rates  for  FY  2008  by  e-mail.  Contact  information  is 
published  in  the  “OPNAV  Planning,  Programming,  Budgeting  and  Execution 
(PPBE)  and  Program  Budget  Information  System  (PBIS)  Courses  for  Fiscal  Year 
2009  memorandum,”  which  is  posted  on  the  N8  homepage  under  “N8  Hot  Topics 
(Public)”  (https://ucstcdom03.ahf.nmci.navy.mil/n8/webbas01  ,nsf/(vwwebpaqe)/ 
home.htm?OpenDocument).  Alternative  cost  measures  are  discussed  in 
Chapter  V,  under  recommendations  for  future  research. 

5.  Safety  Mishap  Data 

The  initial  contact  with  the  Naval  Safety  Center  was  by  telephone.  The 
Naval  Safety  Center,  Afloat,  Surface  Warfare  Division  Website,  at 
http://www.safetycenter.navv.mil/afloat/surface/index.asp,  lists  the  Division 
Head’s  phone  number.  Subsequent  communications  were  via  phone  and  e-mail, 
with  e-mail  being  the  primary  method. 

The  Naval  Safety  Center  provided  an  unclassified  Microsoft  Excel  2003 
document  by  e-mail.  The  Excel  table  gave  the  date  and  description  of  all  mishaps 
onboard  Navy  FFGs,  from  FY  1998  through  FY  2006.  The  columns  of  the  data 
included  Event  Serial  Number,  Ship  UIC,  fiscal  year,  calendar  year  and  month, 

and  mishap  class  (A,  B,  C,  or  D).  These  data  were  used  as  a  measure  of  safety. 
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6.  Manning  and  Personnel  Data 


The  manning,  personnel,  and  SORTS  report  data  were  obtained  from  the 
Defense  Manpower  Data  Center  (DMDC)  via  their  online  Data  Request  System 
(DRS)  at  https://www.dmdc.osd.mil/drs/.  The  initial  request  was  for  all  Enlisted 
Data  Verification  Report  (EDVR)  and  SORTS  reports  on  a  monthly  basis  from 
October  1989  to  October  2006  for  all  Navy  FFGs.  Following  the  online 
submission,  DMDC  made  contact  via  e-mail  and  requested  phone  contact  to 
clarify  the  request.  They  required  a  list  of  UlCs  to  access  the  data  requested. 
Due  to  the  nature  of  the  request,  it  was  split  in  two  datasets  (one  unclassified  and 
one  classified),  and  completed  by  two  different  DMDC  workers. 

DMDC  provided  unclassified  monthly  manning  and  personnel  data  from 
October  1994  to  June  2008,  in  a  comma  separated  text  file.  The  DMDC  database 
did  not  have  monthly  manning  data  prior  to  October  1994.  The  file  contained  six 
columns:  date  (YYMM  format);  rank  (E01-E09,  W01-W05,  001-009  format), 
rating,  (two  to  four  letters  for  enlisted  ratings,  plus  four-digit  NEC  code,  if 
applicable,  and  four-digit  code  for  officer  designators),  years  of  service,  ship  UIC, 
and  count  (number  of  people  with  that  set  of  qualities).  The  file  contained 
905,323  rows  of  data.  In  most  cases,  there  was  only  one  person  on  board  the 
ship  that  month  with  the  exact  same  combination  of  rank,  rate/NEC,  and  years  of 
service.  However,  some  categories,  such  as  undesignated  EOls  with  less  than 
one  year  of  service,  had  as  many  as  12  individuals  with  the  same  characteristics, 
which  is  why  there  was  an  average  of  184  rows  of  manning  and  personnel  data 
per  ship,  per  month. 

7.  Performance  Data 

DMDC  provided  the  SORTS  data,  classified  SECRET,  from  January  1989 

through  September  2006  via  secure  e-mail.  The  Microsoft  Excel  2003  document 

had  12  columns  including  year  (four-digit  format),  month  (two-digit  format), 

component  (active  or  reserve),  and  UIC.  It  also  contained  five  columns  of  that 

month’s  reported  status:  overall  status,  personnel  status,  equipment  condition 
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status,  supply/equipment  status,  and  training  status.  As  discussed  in  the 
literature  review,  each  reported  status  is  inverted  in  order  of  performance  and 
ranges  from  Category  One  (C-1),  best,  to  Category  Five  (C-5),  worst 
performance. 

F.  DATA  CLEANING 

We  obtained  three  and  created  two  separate  sets  of  data  for  each  ship,  for 
each  month,  which  were  merged  to  create  one  data  table.  All  of  the  obtained 
data  files  contained  missing  data  and  duplicate  entries.  This  section  describes 
how  each  of  the  data  tables  was  cleaned  or  created.  The  manning  and  personnel 
data  set  started  as  a  comma  separated  text  file.  The  other  data  sets  provided  or 
created  were  Microsoft  Excel  2003  files.  Unless  otherwise  noted,  all  of  the 
cleaning  and  analysis  was  done  with  the  statistical  software  program  JMP® 
version  7.0.2  (SAS  Institute  Inc.,  2007).  JMP  can  import  a  wide  range  of  files, 
including  comma  separated  text  files  and  Microsoft  Excel  files. 

1 .  Homeport  Data  Table  Creation 

The  UIC  list  was  amended  by  hand  with  Homeport  and  Region  columns. 
This  allowed  for  exploration  of  possible  clustering  of  the  data. 

2.  Mishap  Data  Cleaning 

Some  mishaps  involved  more  than  one  “Mishap  Type”  category, 
e.g.,  “impact  with  object”  and  “flooding”  may  have  occurred  in  the  same  mishap 
and  would  have  created  two  row  entries  for  one  Mishap  Serial  Number.  Since  the 
“Mishap  Type”  and  the  event  description  columns  were  not  of  interest  for  this 
thesis,  these  two  columns  were  deleted.  Next,  the  duplicate  rows  were  deleted. 
This  prevented  duplicate  counting  of  mishaps  when  the  data  were  summed  by 
month  and  ship. 
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3.  Manning  Data  Set  Cleaning 


An  exploratory  pivot  table  of  the  Manning  data  for  one  ship  in  Microsoft 
Excel  2003  returned  unreasonably  large  numbers:  for  total  ship  manning:  over 
400  crewmembers  for  eight  consecutive  months.  Clicking  on  one  of  the  suspect 
cells  in  the  pivot  table  generated  a  new  spreadsheet  with  a  copy  of  the  source 
data.  Sorting  the  data  by  Rate/NEC,  then  by  years  of  service,  revealed  a  pattern 
that  suggested  rows  containing  000  or  E00  in  the  rank  field,  which  represented 
missing  data,  were  duplicate  entries.  Nearly  46,000  out  of  over  905,000  lines  of 
the  Manning  data  appeared  to  be  duplicates.  These  suspect  rows  were  excluded 
from  subsequent  data  analysis. 

4.  Authorized  Manning  and  Personnel  Table  Creation 

A  table  of  the  authorized  manning  and  personnel  levels  was  created  by 
hand.  The  first  column  contained  the  list  of  the  96  months  of  the  final  data  set, 
October  1998  to  September  2006.  The  next  four  columns  contained  the 
“organizational  manpower  requirements”  summary  for  that  month  from  the 
applicable  SMD  obtained  from  NAVMAC.  Table  1  shows  a  condensed  version  of 
the  authorized  manning  table. 


Table  1 .  Historical  Navy  FFG  Authorized  Manning  Levels 


Time  Period 

Officer 

CPO 

Enliste 

d 

Total 

Oct  1998  to  Sep  1999 

16 

17 

188 

221 

Oct  1999  to  Jan  2004 

17 

19 

198 

234 

Feb  2004  to  Oct  2006 

17 

17 

181 

215 

In  December  1997,  the  authorized  crew  level  was  16  officers,  17  Chief 
Petty  Officers  (CPOs)  and  188  “other  enlisted,”  for  a  total  crew  size  of  221 
individuals.  In  September  1999,  the  authorized  crew  was  17  Officers,  19  CPOs, 

and  198  other  enlisted,  for  a  total  crew  of  234  individuals.  In  February  2004,  the 
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authorized  crew  size  was  reduced  to  17  officers,  17  CPOs,  and  171  other 
enlisted,  for  a  total  crew  of  215  individuals.  Section  G.2  of  this  chapter  discusses 
the  2004  reduction  in  crew  size. 

5.  Status  of  Readiness  and  Training  System  (SORTS) 
Data  Cleaning 

An  exploratory  pivot  table  of  the  raw  SORTS  data  in  Microsoft  Excel  2003 
identified  three  possible  issues.  The  table  defined  the  Row  inputs  as  Year  and 
Month,  the  Column  inputs  as  UIC,  and  the  intersections  as  the  “overall”  report. 
The  first  issue  raised  was  that  the  data  set  contained  numerous  holes,  or  missing 
entries.  The  second  issue  was  that  the  data  set  did  not  contain  any  entries  for  a 
few  months;  that  is,  there  were  no  data  for  any  of  the  ships.  The  third  issue  was 
that  some  entries  in  the  table  were  between  6  and  10,  yet  the  raw  data  did  not 
have  any  values  above  5.  Sorting  the  data  by  year,  then  by  month,  then  by  UIC 
uncovered  nearly  four  dozen  duplicate  entries.  The  duplicate  entries  were 
excluded  from  the  data  analysis. 

G.  DEFINITIONS  OF  PREDICTOR  AND  OUTCOME  VARIABLES 

This  section  describes  how  the  variables  were  quantified,  using  the  data 
obtained.  It  is  neither  inclusive  nor  exhaustive  of  data  sources  or  assessment 
methods.  Due  to  data  limitations,  the  time  period  analyzed  was  trimmed  to 
October  1998  through  September  2006. 

Eight  fiscal  years  (FYs),  or  96  months  of  data  on  30  ships,  produced  2,880 
rows  in  the  final  table.  Since  all  of  the  three  data  tables  contained  missing 
entries,  a  two-column  table  was  created  in  Excel  and  imported  into  JMP.  This 
2,880-row  table  contained  every  unique  combination  of  dates  and  UlCs.  Then  we 
merged  the  2,880  table  with  each  of  the  other  data  tables  using  JMP.  By 
including  nonmatches,  JMP  inserted  rows  of  missing  data  as  applicable. 
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1.  Manning 


Manning  is  the  total  number  of  people  assigned  to  the  ship  for  a  given 
month  and  was  used  as  a  substitute  for  manpower.  Manpower  is  the  number  of 
work  hours  that  can  be  performed  by  the  assigned  crew.  Due  to  changes  in 
authorized  manning  levels,  this  thesis  quantified  manning  as  the  percent  of  total 
authorized  crew  assigned  to  a  ship  for  that  month.  The  cleaned  manning  data  set 
for  October  1998  to  October  2006  contained  497,489  rows.  This  data  table  was 
joined  with  the  authorized  manning  data  table.  Summing  by  UIC  and  date 
produced  one  row  per  month,  per  ship.  The  summed  data  table  had  30  missing 
values,  resulting  in  2,850  rows. 

2.  Human  Factors  Engineering  (HFE),  Habitability, 

and  Survivability 

It  was  beyond  the  scope  of  the  current  study  to  obtain  HFE,  habitability,  or 
system  survivability  data.  However,  since  all  ships  were  from  the  “long”  FFG-7 
class,  they  are  assumed  to  have  similar  physical  configurations.  This  study  did 
not  obtain  detailed  data  on  ship  alterations  and  modifications. 

Open  source  information  and  the  reduction  of  authorized  manning  by 
2  CPOs  and  17  junior  enlisted  in  the  January  2004  Ships  Manning  Document 
imply  that  all  30  of  the  Navy’s  FFG-7s  underwent  major  ship  alterations  in  2003. 
The  “Unofficial  U.S.  Navy  site”  states,  “in  2003,  the  Navy  started  to  remove  the 
Mk-13  guided  missile  launchers  from  the  remaining  [FFG-7  Oliver  Hazard] 
PERRY  class  frigates”  (Doehring,  n.d.).  This  concurs  with  the  information  on  the 
“Destroyer  History”  Webpage  that, 

2003  was  a  significant  year  for  the  [FFG-7  Class]  Perrys  in  the  US 
Navy.  Raytheon  terminated  support  for  the  obsolete  SM-1MR 
missile.  [The  US  Navy]  embarked  on  a  program  not  only  to  remove 
the  missile  launchers  from  the  30  long-hull  ships  but  to  modernize 
them,  reducing  their  cost  of  operation  and  extending  their  service 
lives  to  up  to  40  years.  (Destroyer  History  Foundation,  2008) 
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Despite  still  referring  to  them  as  “guided  missile  frigates,”  the  “Official 
Navy  FFG  fact  file”  sheet  does  not  mention  the  ships  ever  having  had  a  missile 
launcher.  “The  guided  missile  frigates  (FFG)  bring  a  short  range  anti-air  warfare 
(AAW)  capability  (provided  by  their  Phalanx  Close-In  Weapon  System)  to  the 
frigate  mission”  (U.S.  Navy,  2007). 

3.  Safety 

Safety  was  quantified  as  the  number  and  frequency  of  mishaps  onboard 
U.S.  Navy  Frigates.  As  discussed  in  the  literature  review,  reports  are  not  required 
for  most  Class  D  mishaps.  Based  on  this  and  preliminary  data  analysis,  safety 
was  quantified  both  with  and  without  Class  D  events. 

Six  safety  variables  were  calculated:  number  of  each  class,  total 
reportable  mishaps  (Class  A,  B,  and  C),  and  total  number  of  mishaps  (Class  A, 
B,  C,  and  D).  The  safety  data  table  was  transformed  using  the  same  method  as 
the  manning  data  table.  This  resulted  in  a  data  table  with  329  rows.  It  was 
merged  with  the  2,880  table  to  align  the  entries  to  the  other  data  tables,  based  on 
ship  UIC  and  month.  Since  ships  do  not  file  monthly  negative  reports,  it  is  not 
possible  to  know  if  any  Class  A,  B,  or  C  reports  are  missing  from  this  data  set. 

4.  Performance 

This  thesis  quantified  performance  as  the  SORTS  reported  levels.  Five 
SORTS  variables  were  provided:  overall  status,  personnel  status,  equipment 
condition  status,  supply/equipment  status,  and  training  status.  As  discussed 
previously,  SORTS  levels  are  low  when  performance  is  high,  i.e.,  a  T  on 
SORTS  corresponds  to  the  maximum  rating  of  performance.  Merging  the  SORTS 
data  table  with  the  2,880  table  filled  in  approximately  400  missing  rows. 
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IV.  RESULTS 


This  chapter  is  divided  into  three  sections.  The  first  section  is  descriptive 
statistics,  the  second  section  is  the  correlation  analysis,  and  the  third  section  is 
an  analysis  of  manning  and  mishap  rankings  across  all  ships.  Three  data  tables 
(manning,  safety,  and  SORTS)  were  merged  into  a  single  data  table  (described 
in  Table  2).  The  columns  of  date  and  UIC  were  used  to  align  the  data.  The  data 
set  contained  2880  rows,  one  for  each  ship  for  each  month,  with  17  columns  per 
row. 

Table  2.  Variables  in  Data  Set 


Date  (month  and  year) 

UIC  (e.g.,  N21032) 

Component  (active  or  reserve) 
Ship  location 

Homeport  (City  and  State) 
Region  (e.g.,  East  Coast) 
SORTS  scores  (1  to  5) 

Overall 

P,  (personnel) 

R,  (equipment) 

S,  (supplies) 

T,  (training) 

Mishaps  (counts) 

Class  A 
Class  B 
Class  C 
Class  D 
Class  A-C 
Class  A-D 

Percent  total  manning _ 


The  date  and  manning  data  were  encoded  as  continuous  variables  in 
JMP.  All  other  variables  were  encoded  as  ordinal  or  categorical  data. 
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A.  DESCRIPTIVE  STATISTICS 
1.  Component 

Of  the  30  ships,  the  SORTS  data  identified  one  ship  as  belonging  to  the 
U.S.  Navy  Reserve  (USNR)  and  the  rest  of  the  ships  as  active  component 
(USN). 


2.  Homeports 

The  Navy’s  FFGs  operate  out  of  six  homeports.  The  largest  concentration 
is  in  Mayport,  Florida,  with  12  ships.  Combined  with  the  six  FFGs  stationed  in 
Norfolk,  Virginia,  there  are  18  FFGs  stationed  on  the  East  Coast  of  the  United 
States.  San  Diego,  California,  has  five  FFGs,  and  Everett,  Washington,  has  three 
FFGs,  for  a  total  of  eight  FFGs  on  the  West  Coast.  Pearl  Harbor,  Hawaii  and 
Yokosuka,  Japan  both  have  two  FFGs,  for  a  total  of  four  FFGs  outside  the 
continental  United  States  (OCONUS).  Table  3  lists  the  number  of  ships  by 
homeport  and  region. 


Table  3.  Navy  FFGs  by  Homeport  and  Region 


HOME  PORT 

8  of 
Ships 

REGION 

8  of 
Ships 

M  ayp  art 

FL 

12 

East 

18 

N  orfok 

VA 

6 

Coast 

San  Diego 

CA 

5 

West 

8 

Cverdt 

WA 

3 

Coast 

PearIH  arbor 

HI 

O 

OCOMUS 

4 

Yokosuka 

Japan 

n 
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3.  SORTS  Report 


The  descriptive  statistics  of  the  SORTS  reports,  classified  as  SECRET, 
are  not  included  in  this  thesis  pending  final  determination  of  classification.  No 
significant  relationships  were  found  between  any  of  the  SORTS  scores  and  any 
of  the  other  variables. 

4.  Mishap  Report  Rates 

Table  4  shows  the  descriptive  statistics  for  the  mishap  reports  for  all  30 
ships  over  96  months.  The  monthly  rates  are  average  mishap  rates  per  ship  per 
month.  The  number  of  mishaps  reported  by  any  ship  over  the  96  months  ranged 
from  three  to  32;  the  average  number  of  mishaps  was  14.17  while  the  most 
common  number  of  reported  mishaps  was  12.  Table  4  includes  statistics  for  the 
subtotal  of  all  reportable  mishaps  (Class  A,  B  and  C).  As  mentioned  in  Chapter  II, 
Literature  Review,  reporting  of  Class  D  mishaps  is  optional. 


Table  4.  Mishap  Statistics  Per  Ship  Over  Eight  FYs 


Minimum  Maximum  Average 

Std  Dev 

Mode 

Total  Monthly 
Rate 

Class  A 

0 

1 

0.20 

0.41 

0 

6 

0.0021 

Class  B 

0 

2 

0.53 

0.73 

0 

16 

0.0056 

Class  C 

1 

13 

5.50 

3.10 

8 

165 

0.0573 

Class  D 

1 

24 

7.93 

6.13 

8 

238 

0.0826 

Class  A-C 

1 

14 

6.23 

3.30 

5 

187 

0.0649 

Class  A-D 

3 

32 

14.17 

6.70 

12 

425 

0.1476 

5.  Manning 


Figure  5  shows  the  percent  manning  over  the  96  months  for  each  of  the 
30  ships.  The  manning  level  for  any  ship  for  any  month  ranged  from  42.74%  to 
121 .40%  with  the  mean  manning  level  at  92.72%  and  median  of  94.87%. 
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The  three  ships  circled  in  red  Figure  5  have  significantly  lower  manning 
levels  than  the  other  27  ships.  One  of  the  three  ships  was  confirmed  to  be 
Reserve  Component  while  the  other  two  are  suspected  to  be  Reserve 
Component.  These  three  ships  were  determined  to  be  sufficiently  different  from 
the  others  and,  therefore,  were  excluded  from  further  analysis. 

Figure  6  shows  the  percent  manning  over  the  96  months  for  each  of  the 
27  ships  believed  to  be  active  component.  The  manning  level  for  any  ship  for  any 
month  ranged  from  64.96%  to  121.40%  with  a  mean  manning  level  of  96.28% 
and  a  median  of  95.93%. 


Figure  6.  Percent  Manning  by  UIC  for  27  Ships 
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B.  CORRELATION  MATRIX 


A  matrix  of  Pearson  correlations  for  percent  manning  and  the  six  safety 
variables  (Classes  A,  B,  C,  D,  Class  A  through  C  and  Class  A  through  D)  was 
calculated  in  Microsoft  Excel  for  the  27  ships  in  the  study  sample.  Table  5  shows 
this  correlation  matrix.  Another  version  of  this  matrix,  including  additional 
exploratory  variables,  is  in  Appendix  A.  All  of  the  correlations  in  Table  5  were 
statistically  significant  at  the  0.05  level.  Not  surprisingly,  the  correlations  between 
several  of  the  mishap  classes  is  fairly  high.  Of  interest  to  this  thesis  is  the 
relationship  between  manning  and  mishaps. 


Table  5.  Correlation  Matrix 


% 


<sv 


% 


% 


°4 


'S', 


<$ 


Percent  Manning 

0.0005  0.0062  0.0295 

0.0253 

0.0295  0.0372 

Class  A 

1.0000  -0.0035  0.0485 

0.0102 

0.2106  0.1327 

Class  B 

1.0000  0.0038 

-0.0165 

0.2557  0.1385 

Class  C 

1.0000 

0.0488 

0.9534  0.6025 

Class  D 

1.0000 

0.0436  0.8064 

Class  A-C 

1.0000  0.6260 

C.  ANALYSIS  OF  RELATIVE  RANKING  BY  SHIP 

To  explore  general  trends  in  manning  and  mishaps,  all  of  the  ships  were 
ranked  from  1  (best),  to  27  (worst),  for  each  month  of  the  study  period  based  on 
percent  of  total  manning  and  number  of  reportable  mishaps.  The  ship  with  the 
highest  percent  manning  for  a  month  was  assigned  a  manning  rank  of  1  while  the 
ship  with  the  lowest  percent  manning  for  a  given  month  was  assigned  a  rank  of 
27.  The  ship  with  the  most  reportable  mishaps  for  a  month  was  assigned  a 
mishap  rank  of  27.  If  a  ship  did  not  have  any  mishaps  for  a  month  it  was 

assigned  a  mishap  rank  of  0,  (i.e.  if  only  three  ships  reported  mishaps  for  a 
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month,  then  the  top  24  ships  were  all  assigned  the  mishap  rank  of  0  and  the 
other  three  were  assigned  mishap  ranks  of  25,  26  and  27).  Table  6  shows  the 
results  of  averaging  each  of  the  two  ranks  over  the  96  months  then  ranking  the 
ships  again  based  on  their  averages.  The  column  on  the  left  is  the  average  of  the 
average  manning  and  average  mishap  ranks  over  96  months  for  each  ship.  The 
right  three  columns  are  the  corresponding  averages  converting  into  a  rank  from  1 
to  27.  The  rows  are  sorted  from  lowest  average  overall  rank  to  highest  average 
overall  rank.  The  three  highest  ranks  are  in  bold  and  the  three  lowest  ranks  are 
underlined  and  in  bold. 


Overall 

Average 

Average 

Overall 

Manning 

Mishap 

Average 

Manning 

Mishap 

Rank 

Rank 

Rank 

3.0573 

5.5625 

0.5521 

1 

1 

3 

3.8073 

5.6667 

1.9479 

2 

2 

22 

4.4844 

7.8750 

1.0938 

3 

3 

10 

5.3125 

9.2708 

1.3542 

4 

4 

13 

5.3542 

10.4271 

0.2813 

5 

5 

2 

5.8906 

10.6979 

1.0833 

6 

6 

9 

6.2604 

10.8646 

1.6563 

7 

7 

17 

6.5990 

11.2292 

1.9688 

8 

8 

23 

6.6354 

12.1667 

1.1042 

9 

9 

12 

6.8490 

12.3229 

1.3750 

10 

10 

14 

6.9479 

12.8438 

1.0521 

11 

11 

6 

7.5781 

14.0833 

1.0729 

12 

12 

8 

8.1563 

14.6354 

1.6771 

13 

13 

19 

8.2396 

16.2188 

0.2604 

14 

18 

1 

8.3594 

16.1667 

0.5521 

15 

17 

4 

8.4219 

16.0104 

0.8333 

16 

15 

5 

8.5104 

15.9271 

1.0938 

17 

14 

11 

8.8594 

16.0938 

1.6250 

18 

16 

16 

9.2135 

17.3750 

1.0521 

19 

22 

7 

9.2917 

17.1979 

1.3854 

20 

20 

15 

9.4688 

17.0104 

1.9271 

21 

19 

20 

9.6667 

17.3958 

1.9375 

22 

23 

21 

9.6979 

17.2396 

2.1563 

23 

21 

24 

10.2813 

17.8125 

2.7500 

24 

25 

26 

10.3906 

17.5313 

3.2500 

25 

24 

27 

10.4531 

19.2500 

1.6563 

26 

27 

18 

10.9219 

19.1250 

2.7188 

27 

26 

25 

Table  6.  Relative  Ranking  of  Ships  by  Manning  and  Mishaps 


As  is  evident  in  Table  6,  ships  with  low  overall  ranks  tend  to  have  both 
relatively  low  manning  levels  and  relatively  high  mishap  rates. 
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Figure  7  compares  manning  rank  over  time  with  the  mishap  rank  over  time 
for  the  ships  with  the  two  highest  and  two  lowest  overall  ranks.  Figure  8  shows 
the  same  data  for  only  two  ships:  the  highest  and  lowest  in  ranking.  Figures  7 
and  8  demonstrate  that  manning  ranks  for  all  of  the  ships  varied  over  time.  No 
ship  was  consistently  ranked  high  or  low  over  the  entire  96  months. 


cdcdoooooo 

oooooooo 

OOOOOOOO 


1st  mishaps 
— —  1st  manning 
-h-  2nd  mishaps 
— -  2nd  manning 
26th  mishaps 
26th  manning 
27th  mishaps 
— -  27th  manning 


Figure  7.  Manning  and  Mishap  Rankings  Over  96  Months  for  the  Top  Two 

and  Bottom  Two  Overall  Ranked  Ships. 
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1st  mishaps 
1st  manning 
27th  mishaps 
— 27th  manning 


Figure  8.  Manning  and  Mishap  Rankings  Over  96  Months  for  The  Top  and 

Bottom  Overall  Ranked  Ships. 


To  determine  whether  relative  ranking  in  manning  level  is  independent  of 
mishap  rankings,  a  nonparametric  correlation  was  performed  in  JMP.  The 
manning  rank  and  mishap  rank  columns  in  Table  6  were  compared  using 
Spearman’s  rho.  The  variables  were  positively  correlated  (Spearman’s  rho  = 
0.4194,  p-value  =  0.0294).  This  finding  shows  that  there  is  a  positive  relationship 
between  a  ship’s  average  manning  rank  and  its  average  mishap  rank  over  the  96 
months.  In  other  words,  there  is  an  indication  of  a  negative  relationship  between 
manning  levels  and  mishap  rates.  Mishap  rates  increase  as  manning  levels  drop. 
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V.  DISCUSSION 


The  first  section  of  this  chapter  discusses  the  three  hypotheses  based  on 
the  results  reported  in  the  previous  chapter.  The  second  section  lists  the 
conclusions  drawn  from  the  analysis  of  relative  rankings.  The  third  section 
discusses  the  implications  of  the  results  with  respect  to  the  NPS  HSI  TST 
computer  program. 

A.  HYPOTHESES 

1.  Hypothesis  One,  Manning  and  Performance 

The  results  failed  to  support  hypothesis  one,  that  there  is  a  positive 
relationship  between  manning  and  SORTS  levels.  There  were  no  significant 
relationships  between  percent  manning  and  any  of  the  five  SORTS  reports. 

2.  Hypothesis  Two,  Manning  and  Safety 

The  results  of  the  rankings  in  Table  6  support  hypothesis  two,  that  there  is 
a  negative  relationship  between  manning  and  mishaps.  There  was  a  statistically 
significant  negative  correlation  between  manning  levels  and  mishap  rates.  Ships 
that  have  relatively  low  percent  manning  levels  also  have  lower  mishap  ranks, 
which  corresponds  to  higher  rates  of  mishaps. 

3.  Hypothesis  Three,  Safety  and  Performance 

The  results  failed  to  support  hypothesis  three,  that  there  is  a  negative 
relationship  between  SORTS  levels  and  mishaps.  There  were  no  statistically 
significant  relationships  between  any  of  the  eight  safety  variables  and  any  of  the 
five  SORTS  report  variables. 
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B.  MANNING  AND  MISHAPS 


Reportable  mishaps  (those  involving  serious  injury  or  damages  over 
$20,000)  appear  to  be  rare.  This  is  good  news  for  the  U.S.  Navy  but  provides  a 
small  data  set  for  analysis.  The  analysis  of  relative  rankings  by  ship  found  a 
statistically  significant  negative  relationship  between  percent  manning  and 
mishap  rates.  The  results  of  this  thesis  support  the  idea  that  there  is  a  negative 
relationship  between  manning  levels  and  mishap  rates,  but  we  suggest  that  more 
detailed  research  is  needed  to  isolate  this  relationship  from  numerous  other 
influences  and  any  noise  that  may  be  present  in  the  data  set.  This  is  discussed 
more  in  Chapter  VI,  Conclusion  and  Recommendations. 

C.  IMPLICATIONS  FOR  NPS  HSI  TST  MODEL 

This  thesis  investigated  two  of  the  22  relationships  in  the  NPS  HSI  TST 
program.  We  found  a  statistically  significant  relationship  between  manning  levels 
and  mishap  rates.  However,  the  analysis  of  ranking  did  not  derive  an  equation  to 
estimate  mishap  rates  based  on  percent  of  manning.  The  average  rate  of 
reportable  mishaps  (Class  A,  B  and  C)  calculated  from  the  safety  data  provided 
is  0.06  mishaps  per  ship  per  month.  If,  for  example,  the  mishap  rate  for  USN 
FFGs  is  not  significantly  influenced  by  the  HFE  level,  then  the  average  rate  can 
be  used  to  define  the  relationship  between  HFE  and  Safety  in  the  NPS  HSI  TST. 
However,  defining  the  mishap  rate  as  a  constant  would  make  it  irrelevant  in 
tradeoff  analysis. 

Although  this  was  not  pursued,  the  personnel  costs  provided  by  N80, 
$59,139  per  year  per  enlisted  and  $123,269  per  year  per  officer,  could  be  used  to 
define  the  relationships  of  cost  with  manpower  and  personnel. 
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VI.  CONCLUSION  AND  RECOMMENDATIONS 


The  first  section  of  this  chapter  discusses  how  finding  no  significant 
relationship  between  many  of  the  variables  can  justify  a  reduced  sample  size  for 
future  studies.  The  second  section  suggests  ways  to  improve  the  data  collection 
and  analysis  methods  for  each  of  the  HSI  domains.  The  third  section  suggests 
two  possible  directions  to  expand  data  collection  beyond  USN  FFGs.  The  fourth 
section  restates  the  implications  of  the  results  of  this  thesis. 

A.  CONSISTENCY 

No  practically  significant  differences  were  found  in  manning,  personnel, 
performance  or  safety  based  on  date,  ships,  homeport,  or  region.  For  example, 
there  is  no  evidence  that  ships  in  Hawaii  are  more  likely  to  report  a  class  D 
mishap  than  ships  in  Norfolk,  Virginia.  This  means  that  data  collected  on  a 
sample  of  ships  should  be  applicable  to  all  Navy  Frigates.  However,  there  were 
significant  differences  in  manning  levels  between  one  ship  that  was  classified  as 
a  Navy  Reserve  ship  in  the  SORTS  data  and  29  of  the  ships  classified  as  Navy 
Active  component.  Two  additional  ships  in  the  sample  appeared  to  be  Navy 
Reserve  ships.  The  low  manning  level  for  these  three  ships  may  be  due  to 
exclusion  of  activated  U.S.  Navy  Reserve  personnel  in  the  database. 

B.  IMPROVEMENTS  TO  HSI  DATA 

This  section  discusses  possible  improvements  to  the  types  of  data 
collected,  and  analysis  methods  utilized,  to  explore  HSI  in  the  U.S.  Navy.  Future 
research  may  benefit  from  more  detailed  data  collection.  This  thesis  chose 
breadth  (eight  fiscal  years)  over  depth  of  data  (e.g.,  average  hours  spent  on 
watch  over  one  month  of  underway  operations).  The  data  analyzed  for  this  thesis 
were  insufficient  to  validate  all  the  relationships  in  the  notional  trade  space  tool. 
However,  empirical  evidence  was  found  to  support  a  negative  relationship 
between  manning  and  mishap  rates.  This  thesis  did  not  provide  substitutes  for 
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missing  data.  The  missing  SORTS  data  could  be  estimated  based  on  trends  for 
the  ship,  but  this  may  be  less  accurate,  especially  if  several  reports  in  a  row  are 
missing. 

1.  Manpower 

The  historical  reports  collected  by  the  U.S.  Navy  do  not  record  manpower 
data.  Therefore,  manning  was  used  as  a  surrogate  for  manpower.  The  data 
collected  could  not  distinguish  between  a  crew  that  works  40  hours  a  week 
(which  is  a  common  practice  while  in  port)  from  a  crew  that  works  over  80  hours 
a  week  (common  on  deployments)  to  maintain  an  overall  SORTS  score  of 
Category  1.  A  better  approximation  would  be  to  compare  actual  workload, 
collected  onboard  a  sampling  of  ships,  with  the  prescribed  workload  data 
contained  in  the  SMDs  provided  by  NAVMAC.  A  prospective  study  could  be 
designed  to  do  this  type  of  analysis.  Collecting  these  data  retrospectively  is 
virtually  impossible.  However,  past  manpower  levels  could  be  estimated  by 
collecting  and  analyzing  the  following:  ship’s  schedules,  typical  watch  stations 
manned  based  on  ship’s  schedule,  personnel  assigned,  and  maintenance  jobs 
completed.  Other  aspects  of  manpower,  such  as  unit  cohesion  and  morale,  could 
be  estimated  based  on  crew  turnover  rates. 

2.  Personnel  and  Training  Variables 

Eight  personnel  variables  were  analyzed  but  were  deemed  inadequate  to 
test  relationships  between  personnel  and  the  other  HSI  domains.  The  variables 
were  percent  of  junior  enlisted  (E01-E06),  percent  of  Chief  Petty  Officers  (CPOs), 
(E07-E09),  percent  of  officers,  percent  of  personnel  with  a  Navy  Enlisted 
Classification  (NEC)  code,  and  four  summations  of  years  of  service.  Chief 
Warrant  Officers  (W01-W05)  were  included  in  the  officer  category.  Officer 
designators  were  evaluated  as  personnel  without  an  NEC. 

This  thesis  proposed  using  NECs,  rank,  and  years  of  service  as  personnel 
and  training  measures.  The  SMDs  provided  by  NAVMAC  and  the  data  provided 
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by  DMDC  both  included  the  same  rank,  rate,  and  NEC  information.  One  could 
compare,  by  billet,  the  assigned  personnel  with  the  authorized  personnel  list. 
However,  this  method  was  beyond  the  scope  of  this  thesis.  A  billet  comparison  is 
just  one  way  to  measure  the  personnel  and  training  levels.  Additional  information 
sources  could  include  an  individual’s  aptitude  test  scores  at  enlistment,  grade  or 
class  standing  at  service  schools,  and  advancement  exam  scores. 

3.  Training 

Based  on  the  data  collected,  it  was  not  possible  to  quantify  training. 
Training  could  be  quantified  as  the  activities  that  increase  a  unit’s  level  of 
personnel.  In  this  way,  a  ship  that  sends  a  crewmember  to  school  to  obtain  an 
NEC,  achieves  the  same  result  as  a  ship  that  detaches  an  unqualified  person 
and  obtains  a  new  person  with  the  desired  NEC.  In  both  cases,  the  manning 
level  stayed  constant,  while  one  measure  of  personnel  increased. 

4.  Schedule 

Schedule,  as  defined  in  the  program  acquisition  process,  normally  only 
applies  to  delivery  of  new  systems.  Generally,  it  is  not  applicable  to  the 
evaluation  of  legacy  systems,  unless  the  system  is  completing  a  modernization 
program.  Scheduling  data,  such  as  operational  tempo  and  unit  availability  for 
tasking,  is  performance  related  and  could  be  used  in  similar  efforts  in  the  future. 

5.  Human  Factors  Engineering 

Due  to  the  many  similarities  within  the  ‘long’  FFG  class,  HFE  is  difficult  to 
assess  within  this  community.  HFE  may  be  more  useful  for  an  analysis  that 
includes  several  different  classes  (e.g.,  cruisers,  destroyers  and  frigates). 
Several  methods  could  test  if  HFE  is  a  significant  factor,  to  include  determining 
the  exact  time  period  around  2003  when  each  ship  had  its  missile  launcher 
removed  and  other  major  upgrades  performed.  One  could  also  obtain  detailed 
information  on  every  ship  alteration  and  modification,  such  as  the  addition  of  a 
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new  optic  system,  or  upgrades  to  a  computer  system.  It  would  be  possible  to 
obtain  data  for  the  decommissioned  ‘short’  FFGs  and  investigate  possible  effects 
between  these  two  distinct  categories  of  ship  platforms. 

6.  Habitability,  and  Survivability 

As  mentioned  in  Chapter  III,  Section  G,  this  study  did  not  obtain  any 
detailed  human  factors  engineering,  habitability  or  system  survivability  data.  The 
NPS  HSI  TST  model  theorizes  that  decisions  made  regarding  manpower, 
personnel,  training  and  HFE  determine  safety,  habitability  and  system 
survivability.  Just  as  training  changes  the  level  of  personnel,  changes  in  HFE 
impact  the  level  of  habitability  and  survivability.  Therefore,  any  efforts  to  improve 
the  measure  of  HFE  should  improve  the  measures  of  habitability  and  survivability 
as  well. 

7.  Risk/Safety  Variable 

The  results  of  this  thesis  identify  safety  as  the  variable  that  would  most 
benefit  from  improved  data  collection  and  improved  analysis.  Due  in  part  to  the 
infrequent  rate  of  all  mishaps,  this  thesis  did  not  analyze  causal  factors  for  any  of 
the  reported  mishaps  (e.g.,  mishaps  due  to  heavy  seas  while  underway  were  not 
evaluated  separately  from  mishaps  during  command  sponsored  sports  activities 
ashore).  Board  categories,  included  in  the  safety  data  from  the  Naval  Safety 
Center,  could  be  used  to  isolate  only  those  mishaps  suspected  of  being  caused 
by  manning  or  training  factors.  A  better  approach  would  be  a  detailed  qualitative 
analysis  of  the  full  written  mishap  report  for  each  mishap.  Including  minor  injury 
and  illness  rates  to  future  studies  would  increase  the  volume  and  sensitivity  of 
safety  data  collected. 

8.  Cost  Variable 

Cost  of  ownership  was  not  used  in  the  data  analysis,  but  some 
consideration  was  given  to  how  to  address  the  cost  of  a  system.  It  is  possible  to 
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quantify  manning  and  personnel  costs  as  the  FY  2008  Active  Duty  Programming 
Rates  of  $123,269  per  year  for  officers  and  $59,139  per  year  enlisted.  This  thesis 
did  not  collect  any  cost  estimates  for  training,  HFE,  acquisition,  operations  or 
maintenance. 

Developing  an  accurate  method  to  determine  total  system  cost  is  a  goal 
worthy  of  its  own  study.  One  challenge  in  capturing  the  true  cost  of  any  DoD 
system  is  in  defining  the  boundaries.  In  addition  to  payroll,  a  ship’s  costs  could 
include  fuel,  food,  repair  parts,  shore  based  maintenance  facilities,  and  medical 
care.  For  more  information  on  alternate  cost  estimation  methods  see  Naval 
Center  for  Cost  Analysis  (NCCA)  ‘NCCA  Cost  Tools’  webpage  at 
http://www.ncca.navv.mil/services/costtools.cfm. 

This  thesis  recommends  quantifying  manning  and  personnel  cost  as  the 
Active  Duty  Programming  rate  for  a  single  fiscal  year.  An  alternative  would  be  to 
obtain  the  manpower  programming  rates  for  each  FY  from  the  Program  Budget 
Information  System  (PBIS)  database  or  from  N80,  Programming  Division. 
Separate  rates  for  activated  reservists  are  also  available. 

9.  Performance  Variables 

All  of  the  performance  data  used  in  this  thesis  were  based  on  the  SORTS 
data  reported  by  each  ship.  The  results  of  this  thesis  suggest  that  the  SORTS 
scores  are  insensitive  to  the  variables  quantified.  The  SORTS  scores  are  also 
somewhat  subjective  and  confounded  by  the  independent  variables  (e.g.,  if  a 
ship  does  not  have  a  critical  NEC  billet  filled,  it  has  to  report  a  lower  personnel 
level).  Other  performance  measures  that  could  be  included  in  future  studies  are 
externally  graded  ship  exercises,  personnel  reenlistment  rates,  time  to  earn 
watch  station  qualifications,  operational  tempo,  and  unit  availability  for  tasking. 

C.  ADDITIONAL  APPLICATIONS 

The  work  of  this  thesis  could  be  expanded  to  include  additional  ships  that 

are  similar  to  the  USN  Frigates,  (e.g.  Cruisers  and  Destroyers)  to  permit  greater 
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pooling  of  data  or  it  could  be  expanded  to  a  variety  of  diverse  naval  platforms 
(e.g.,  amphibious  ships,  Submarines,  or  Aircraft  Carriers). 

D.  CONCLUSION 

HSI,  though  required  by  law,  is  often  confused  as  just  HFE  or  just  SE. 
There  remains  a  need  for  shared  representations  of  the  HSI  trade  space  to 
facilitate  communication  between  what  are  often  isolated  domains.  While  the 
data  collected  for  this  thesis  were  insufficient  to  validate  all  the  relationships  in 
one  notional  trade  space  tool,  finding  empirical  evidence  of  a  negative 
relationship  between  manning  levels  and  mishap  rates  is  a  substantial  step 
forward.  Just  as  this  thesis  advanced  previous  work,  further  research  is  needed 
to  develop  the  NPS  HSI  Trade  Space  Tool  to  maturity. 
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APPENDIX  A 


CORRELATION  MATRIX 


This  appendix  contains  an  exploratory  correlation  matrix  of  16  continuous 
variables  from  the  25  original  variables. 
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APPENDIX  B.  DODI  5000.2  ENCLOSURE  7 


E7.  ENCLOSURE  7,  HUMAN  SYSTEMS  INTEGRATION  (HSI) 

E7.1.  General.  The  PM  shall  have  a  comprehensive  plan  for  HSI  in  place  early  in 
the  acquisition  process  to  optimize  total  system  performance,  minimize  total 
ownership  costs,  and  ensure  that  the  system  is  built  to  accommodate  the 
characteristics  of  the  user  population  that  will  operate,  maintain,  and  support  the 
system.  HSI  planning  shall  be  summarized  in  the  acquisition  strategy  and 
address  the  following: 

E7.1.1.  Human  Factors  Engineering.  The  PM  shall  take  steps  (e.g.,  contract 
deliverables  and  Government/contractor  IPT  teams)  to  ensure  human  factors 
engineering/cognitive  engineering  is  employed  during  systems  engineering  over 
the  life  of  the  program  to  provide  for  effective  human-machine  interfaces  and  to 
meet  HSI  requirements.  Where  practicable  and  cost  effective,  system  designs 
shall  minimize  or  eliminate  system  characteristics  that  require  excessive 
cognitive,  physical,  or  sensory  skills;  entail  extensive  training  or  workload¬ 
intensive  tasks;  result  in  mission-critical  errors;  or  produce  safety  or  health 
hazards. 

E7.1.2.  Personnel.  The  PM  shall  work  with  the  personnel  community  to  define 
the  human  performance  characteristics  of  the  user  population  based  on  the 
system  description,  projected  characteristics  of  target  occupational  specialties, 
and  recruitment  and  retention  trends.  To  the  extent  possible,  systems  shall  not 
require  special  cognitive,  physical,  or  sensory  skills  beyond  that  found  in  the 
specified  user  population.  For  those  programs  that  require  skill  requirements  that 
exceed  the  knowledge,  skills,  and  abilities  of  current  military  occupational 
specialties  or  that  require  additional  skill  indicators  or  hard-to-fill  military 
occupational  specialties,  the  PM  shall  consult  with  personnel  communities  to 
identify  readiness,  personnel  tempo  (PERSTEMPO),  and  funding  issues  that 
impact  program  execution. 

E7.1.3.  Habitability.  The  PM  shall  work  with  habitability  representatives  to 
establish  requirements  for  the  physical  environment  (e.g.,  adequate  space  and 
temperature  control)  and,  if  appropriate,  requirements  for  personnel  services 
(e.g.,  medical  and  mess)  and  living  conditions  (e.g.,  berthing  and  personal 
hygiene)  for  conditions  that  have  a  direct  impact  on  meeting  or  sustaining  system 
performance  or  that  have  such  an  adverse  impact  on  quality  of  life  and  morale 
that  recruitment  or  retention  is  degraded. 

E7.1.4.  Manpower.  In  advance  of  contracting  for  operational  support  services, 
the  PM  shall  work  with  the  manpower  community  to  determine  the  most  efficient 
and  cost-effective  mix  of  DoD  manpower  and  contract  support.  Once  the 
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Manpower  Estimate  is  approved  by  the  DoD  Component  manpower  authority,  it 
shall  serve  as  the  authoritative  source  for  reporting  manpower  in  other  program 
documentation. 

E7.1.5.  Training.  The  PM  shall  work  with  the  training  community  to  develop 
options  for  individual,  collective,  and  joint  training  for  operators,  maintainers  and 
support  personnel  and,  where  appropriate,  base  training  decisions  on  training 
effectiveness  evaluations.  The  PM  shall  address  major  elements  of  the  training 
system  described  in  DoD  Directive  1430.13,  reference  (bd),  and  place  special 
emphasis  on  options  that  enhance  user  capabilities,  maintain  skill  proficiencies, 
and  reduce  individual  and  collective  training  costs.  The  PM  shall  develop  training 
system  plans  to  maximize  the  use  of  new  learning  techniques,  simulation 
technology,  embedded  training,  and  instrumentation  systems  that  provide 
anytime,  anyplace  training  and  reduce  the  demand  on  the  training  establishment. 
Where  possible,  the  PM  shall  maximize  the  use  of  simulation-supported 
embedded  training,  and  the  training  systems  shall  fully  support  and  mirror  the 
interoperability  of  the  operational  system.  For  training  programs  that  require 
training  infrastructure  modifications,  the  PM  shall  identify  technology,  schedule, 
and  funding  issues  that  impact  program  execution. 

E7.1.6.  Environment,  Safety  and  Occupational  Health  (ESOH).  As  part  of  risk 
reduction,  the  PM  shall  prevent  ESOH  hazards  where  possible,  and  shall 
manage  ESOH  hazards  where  they  cannot  be  avoided.  The  acquisition  strategy 
shall  incorporate  a  summary  of  the  Programmatic  ESOH  Evaluation  (PESHE), 
including  ESOH  risks,  a  strategy  for  integrating  ESOH  considerations  into  the 
systems  engineering  process,  identification  of  ESOH  responsibilities,  a  method 
for  tracking  progress,  and  a  compliance  schedule  for  NEPA  (42  U.S.C.  4321- 
4370d  and  Executive  Order  12114,  references  (x)  and  (az)).  During  system 
design,  the  PM  shall  document  hazardous  materials  used  in  the  system  and  plan 
for  the  system’s  demilitarization  and  disposal.  The  CAE  (or  for  joint  programs,  the 
CAE  of  the  Lead  Executive  Component)  or  designee,  is  the  approval  authority  for 
system-related  NEPA  and  E.O.  12114  documentation.  For  acceptance  of  ESOH 
mishap  risks  identified  by  the  program,  the  CAE  is  the  acceptance  authority  for 
high  risks,  PEO-level  for  serious  risks,  and  the  PM  for  medium  and  low  risks  as 
defined  in  the  industry  standard  for  system  safety. 

E7.1.7.  Survivability.  For  systems  with  missions  that  might  require  exposure  to 
combat  threats,  the  PM  shall  address  personnel  survivability  issues  including 
protection  against  fratricide,  detection,  and  instantaneous,  cumulative,  and 
residual  nuclear,  biological,  and  chemical  effects;  the  integrity  of  the  crew 
compartment;  and  provisions  for  rapid  egress  when  the  system  is  severely 
damaged  or  destroyed.  The  PM  shall  address  special  equipment  or  gear  needed 
to  sustain  crew  operations  in  the  operational  environment. 

Last  Modified  on:  12/17/2004  (DODI  5000.2,  2004,  p.  E7). 
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